-- Logs begin at Tue 2022-09-20 04:12:53 UTC, end at Tue 2022-09-20 05:15:10 UTC. -- Sep 20 04:37:27.811173 dv-cinder-756710-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 20 04:37:27.821025 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 20 04:37:27 2022] *** Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: nodename: dv-cinder-756710-7-iscsi Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: machine: x86_64 Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: clock source: unix Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: pcre jit disabled Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: detected number of CPU cores: 4 Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: current working directory: / Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: detected binary path: /usr/bin/uwsgi-core Sep 20 04:37:27.823064 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: your processes number limit is 47680 Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: your memory page size is 4096 bytes Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: detected max file descriptor number: 2048 Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: lock engine: pthread robust mutexes Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: thunder lock: enabled Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 20 04:37:27.823677 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 20 04:37:27.862800 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: Python main interpreter initialized at 0x55ef6b8b79c0 Sep 20 04:37:27.862800 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: python threads support enabled Sep 20 04:37:27.862800 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: your server socket listen backlog is limited to 100 connections Sep 20 04:37:27.862800 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: your mercy for graceful operations on workers is 90 seconds Sep 20 04:37:27.863156 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: mapped 403077 bytes (393 KB) for 2 cores Sep 20 04:37:27.863249 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: *** Operational MODE: preforking *** Sep 20 04:37:27.863486 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: *** uWSGI is running in multiple interpreter mode *** Sep 20 04:37:27.863486 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: spawned uWSGI master process (pid: 87782) Sep 20 04:37:27.863685 dv-cinder-756710-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 20 04:37:27.865536 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: spawned uWSGI worker 1 (pid: 87797, cores: 1) Sep 20 04:37:27.865536 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: spawned uWSGI worker 2 (pid: 87798, cores: 1) Sep 20 04:37:27.865536 dv-cinder-756710-7-iscsi devstack@keystone.service[87782]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 20 04:37:32.135283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 20 04:37:32.135283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: from cryptography.utils import int_from_bytes Sep 20 04:37:32.136043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 20 04:37:32.136043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: from cryptography.utils import int_from_bytes Sep 20 04:37:32.136462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 20 04:37:32.136462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: from cryptography.utils import int_from_bytes Sep 20 04:37:32.136579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 20 04:37:32.136579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: from cryptography.utils import int_from_bytes Sep 20 04:37:32.185754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 20 04:37:32.185948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 20 04:37:32.186047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 20 04:37:32.186140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] command line args: [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 20 04:37:32.186218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 20 04:37:32.186339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] command line args: [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 20 04:37:32.186425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 20 04:37:32.186567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ================================================================================ {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 20 04:37:32.186644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 20 04:37:32.186779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ================================================================================ {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 20 04:37:32.186862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] admin_token = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] admin_token = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] config_dir = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] config_dir = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] config_source = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] debug = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] config_source = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.187791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] debug = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.188915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_config_append = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_config_append = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.189880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_dir = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_dir = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_file = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_file = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.190984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.191964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.192925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.193909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] publish_errors = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] publish_errors = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.194937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.195017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.195088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.195169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] standard_threads = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.195332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] standard_threads = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.195973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] use_journal = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] use_json = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] use_journal = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] use_stderr = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] use_json = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] use_syslog = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] use_stderr = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.196975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] use_syslog = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.197110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.197204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 20 04:37:32.197288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.197464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.197542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.197707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.197790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.197949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.198929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.199980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.200966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.201862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.202924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.203867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.204999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.205954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.206977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.207885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.208851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.209969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.210981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.211898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.212885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.213999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.214956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.215986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.216953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.217905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.218879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.219966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.220915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.221943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.external = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.222923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.external = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.password = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.223907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.password = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] auth.token = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] auth.token = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.224869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.225977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.226063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.226218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.226296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.228913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.229941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.230950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.231985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.232972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.233996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.234958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.235881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.236891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.237854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.238942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.239975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.240951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.241932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.242031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.242635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.242787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.242874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.243865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.244897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.245887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.246994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.247982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.248924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.249957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.250962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.251966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.252880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.253882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.254899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.255916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.256950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] role.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.257839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.258480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] role.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.258603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] role.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.258732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.258859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] role.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.258979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.259981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.260934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.261972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.262971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.263994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.264889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.265907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.266920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.267937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.268994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.269949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.connection = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.connection = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.270965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.271975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.272954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.273949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.274043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87797) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 20 04:37:32.274130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.274347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 20 04:37:32.274521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87798) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 20 04:37:32.303121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.303349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.303449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.303572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.303657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.303849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.312433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.312727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.340755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.340985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.341326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.341564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.394470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.394846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.443047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.443285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.443463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.443705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.598803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.599912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.600004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.600135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.623673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.623910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.624991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.625072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.625209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.625428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.636047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.636354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.648934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.649875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87797) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.650668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87798) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 20 04:37:32.667148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.667358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 20 04:37:32.667627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.667709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.667843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 20 04:37:32.667940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 20 04:37:32.668146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.668227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.668378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 20 04:37:32.668460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.668685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.668781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.668945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.669029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.669204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.669465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.678441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.678726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.679000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.679081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.679285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.679365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.680661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.680661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.687011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.689924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.690215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.690502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.690788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.691073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.691234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.691337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.691610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.691886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.692158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.692434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.692731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.695961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.696256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.696547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.696879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.697176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.697449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.697730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.698013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.698291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.698571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.698845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.715260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.716739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.717030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.721587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.721684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.721922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.722207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.723080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.723380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.728898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.729166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.729395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.729478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.729876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.735053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.735356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.735649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.735941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.736224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.736507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.736806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.737079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.738281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.738579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.745021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.745328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.745627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.745926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.746214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.746501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.746783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.747063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.752942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.753227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.757078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.760346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.760665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.762463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.762751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.764852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.766572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.768017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.769789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.770095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.771238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.771542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.771832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.772122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.772415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.772729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.774681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.777880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.781151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.781460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.781754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.782047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.782344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.782632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.788566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.788881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.789667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.789935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.796720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.798258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.798841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.799133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.799405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.799674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.799779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.800075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.802845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.803136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.803422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.803713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.804115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.807256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.808830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.810381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.810666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.813376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.813669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.813972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.814260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.814656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.815697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.815988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.816272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.816558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.816967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.817250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.817531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.825808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.826099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.826385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.826675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.827060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.827348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.827632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.832775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.835677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.840534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.842036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.843501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.843788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.844195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.844327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.847277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.850187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.850540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.852256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.852525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.853771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.854070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.854471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.855423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.855822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.856112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.856395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.860864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.862336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.864134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.864431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.864746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.865276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.865681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.865974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.866258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.871581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87797) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.871871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.872153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.872429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.872730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.873015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.873290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.873565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.873835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.874164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.874918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.875202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87797) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.881542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87798) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 20 04:37:32.881837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.882129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.882414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.882717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.882994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.883272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.883571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.883848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.884124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.884414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.884718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.885021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.885334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87798) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 20 04:37:32.909609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef6b8b79c0 pid: 87797 (default app) Sep 20 04:37:32.912814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c290104-339d-4d43-b01d-e1d72fdee172 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:32.913103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c290104-339d-4d43-b01d-e1d72fdee172 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:32.913376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c290104-339d-4d43-b01d-e1d72fdee172 None None] PATH_INFO: `/v3` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:32.918046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Tue Sep 20 04:37:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:32.920621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ef6b8b79c0 pid: 87798 (default app) Sep 20 04:37:39.767269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:39.767813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:39.767813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:39.769146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:37:39.777069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:39.777656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:39.777656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:39.846804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: INFO dbcounter [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] Registered counter for database keystone Sep 20 04:37:39.866102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG oslo_db.sqlalchemy.engines [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87798) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 20 04:37:39.872344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writer thread running {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 20 04:37:40.058818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG passlib.handlers.bcrypt [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87798) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 20 04:37:40.059120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG passlib.handlers.bcrypt [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87798) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 20 04:37:40.122433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:40.124067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:40.251429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1/3] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 20 04:37:40.681276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:40.753134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: INFO dbcounter [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] Registered counter for database keystone Sep 20 04:37:40.776053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG oslo_db.sqlalchemy.engines [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87797) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 20 04:37:40.782734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writer thread running {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 20 04:37:40.899874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1e5d5-6ac9-4acc-b6c4-722b84846b7c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:40.904066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:40.904707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:40.904985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:40.905253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:40.913308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:40.965828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 20 04:37:40.966114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d0e8e-5612-416a-96ec-5c886e1ef1d6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:40.975296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 3/4] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Sep 20 04:37:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 04:37:40.981860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:40.999050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de061417-8f8d-4d37-9370-1c8ea816158d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:41.003431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:41.004690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:41.004975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:41.005246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] PATH_INFO: `/v3/projects/612c023f1ed44b9f936be9c752c292ec` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:41.006676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] RBAC: Authorizing `identity:get_project(project_id=612c023f1ed44b9f936be9c752c292ec)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:41.060814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING keystone.common.rbac_enforcer.enforcer [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 20 04:37:41.061159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d6546-24b7-4d0c-b887-e47fbff2e603 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:41.065644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 2/5] 10.222.0.27 () {56 vars in 1207 bytes} [Tue Sep 20 04:37:40 2022] GET /identity/v3/projects/612c023f1ed44b9f936be9c752c292ec => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:43.179317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8b7c-44a5-4040-a860-f4c27fd9f682 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:43.179710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8b7c-44a5-4040-a860-f4c27fd9f682 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:43.179901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8b7c-44a5-4040-a860-f4c27fd9f682 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:43.184010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 4/6] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:43.189438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:43.189842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:43.189981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:43.210471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:43.212633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:43.220330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 3/7] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 20 04:37:43.487509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8b7c-44a5-4040-a860-f4c27fd9f682 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:43.499462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8b7c-44a5-4040-a860-f4c27fd9f682 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:43.503664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:43.504254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:43.504534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:43.504837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:43.512359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:43.514165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b6513-3f1c-47df-8fb6-7fea347ed1af admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:43.522715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 5/8] 10.222.0.27 () {58 vars in 1324 bytes} [Tue Sep 20 04:37:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:37:43.528777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:43.537669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b42c0-c023-4175-9914-5b78c86bd6b6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:43.542625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:43.543239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:43.543516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:43.543784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:43.545108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:43.546910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7611-9055-4d8e-996c-23bba551947a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:43.548397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 4/9] 10.222.0.27 () {56 vars in 1198 bytes} [Tue Sep 20 04:37:43 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:45.781752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f9cf-9639-49cc-91d7-3173a4ac0677 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:45.782323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f9cf-9639-49cc-91d7-3173a4ac0677 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:45.782323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74f9cf-9639-49cc-91d7-3173a4ac0677 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:45.783618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:45.789053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b81804-b18e-4337-9ffd-b0bf61195f53 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:45.789956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b81804-b18e-4337-9ffd-b0bf61195f53 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:45.790109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b81804-b18e-4337-9ffd-b0bf61195f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:45.807889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b4b81804-b18e-4337-9ffd-b0bf61195f53 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:45.809112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b4b81804-b18e-4337-9ffd-b0bf61195f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:45.888502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 5/11] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:46.164140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74f9cf-9639-49cc-91d7-3173a4ac0677 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.170620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d68e6-7a25-4ee4-aa17-aeca94b4f07a None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.170906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d68e6-7a25-4ee4-aa17-aeca94b4f07a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.171181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d68e6-7a25-4ee4-aa17-aeca94b4f07a None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.172498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 6/12] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:37:46.177492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.177778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.178170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.200711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:46.201929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:46.219496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.272231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:37:46.313947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74f9cf-9639-49cc-91d7-3173a4ac0677 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:46.347846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:46.348826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.349232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.349629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.407111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.441781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:46.443901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:46.537282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.576444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 7/14] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:37:46.582682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.650293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491cf2e-16bd-4713-bf6b-d1ada408a54e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:46.659188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.683807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:46.684445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.684735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667226e1-460d-4ce1-8a02-51d0a2ac4393 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:46.684856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.685027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.696667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:37:46.705447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:37:46.706422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:37:46.707287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:37:46.713038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.718433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:46.719280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.719563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.719831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.727758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.739216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:46.740727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:37:46.759543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:37:46.761526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:37:46.763320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:37:46.763320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:37:46.763320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:37:46.763320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 8/16] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:37:46.766370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.826739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9f38-b688-49d7-8f49-1ad52cb295ca None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:46.862004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:46.862624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.862910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.863185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.863866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:46.864168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:46.865591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:46.872432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08611aad-e684-4f1a-a743-a5b3524b0fb0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:46.878256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:46.884514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:46.898287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.907528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:46.908147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:46.908427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:46.908728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:46.909443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:46.909697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:46.911087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:46.928623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:46.965587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:46.993576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dfa92-e24c-4ec4-9898-0c2c64f4988c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:47.028545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:47.029222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:47.029510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:47.029788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:47.030969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:47.032772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:47.034386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:37:46 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:47.040588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:47.155225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-54e9-4d7b-a2d2-94235ee7f84a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:47.189825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:47.190404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:47.191091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:47.191091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:47.192421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:47.194152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a9424-7f31-4fda-9422-7a14c45da8c0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:47.195318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:47.201622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:47.312254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddf479-9168-4831-8fa9-eb6f754e4fee None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:47.352183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:47.353192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:47.353701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:47.353974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:47.354706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.domain.id=default, user.id=095a92acb2e04f44b21db4de77e2a16a) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:47.355014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:47.356453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62d7d-5221-4dd9-ae65-187e29627ab4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:47.364243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Sep 20 04:37:47 2022] GET /identity/v3/role_assignments?user.id=095a92acb2e04f44b21db4de77e2a16a&scope.domain.id=default&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:49.358739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a5a9-03e1-4fea-8845-b0b627fa0fd1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:49.359241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a5a9-03e1-4fea-8845-b0b627fa0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:49.359340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9a5a9-03e1-4fea-8845-b0b627fa0fd1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:49.360632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:49.365911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746479-414d-40df-9376-149d0e6c309a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:49.366334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746479-414d-40df-9376-149d0e6c309a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:49.366512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6746479-414d-40df-9376-149d0e6c309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:49.384844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:49.393661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e6746479-414d-40df-9376-149d0e6c309a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:49.395966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e6746479-414d-40df-9376-149d0e6c309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:49.488381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:49.938160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406a31-0391-4885-9301-7871d2bf856a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:49.938896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406a31-0391-4885-9301-7871d2bf856a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:49.939399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec406a31-0391-4885-9301-7871d2bf856a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:49.940986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 12/24] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:49.949201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40437e5f-ba09-4347-a1a3-10141f0823a4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:49.949802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40437e5f-ba09-4347-a1a3-10141f0823a4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:49.950306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40437e5f-ba09-4347-a1a3-10141f0823a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:49.969924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-40437e5f-ba09-4347-a1a3-10141f0823a4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:49.971452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-40437e5f-ba09-4347-a1a3-10141f0823a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:50.030939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec406a31-0391-4885-9301-7871d2bf856a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.066769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:50.124779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:50.170422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db16d0-2924-4327-9947-96130d346a2e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.170708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db16d0-2924-4327-9947-96130d346a2e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.171146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db16d0-2924-4327-9947-96130d346a2e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.174560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 14/26] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:50.180031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.180430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.180848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.189361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec406a31-0391-4885-9301-7871d2bf856a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:50.211794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:50.214699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:50.255573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:50.256206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.256481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.256775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.277324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:50.279529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:37:50.309183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:37:50.310113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:37:50.310948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:37:50.310948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:37:50.310948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:37:50.310948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 13/27] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:37:50.324393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.336998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:50.349283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 15/28] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:37:50.480682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.487754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c2ed3-025d-4835-8938-1c8b5b99378a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:50.495940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:50.525083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:50.525695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.525984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.526236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.526873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:50.527172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:50.528540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:50.541155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 14/29] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:50.546710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.610715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc3e4-e666-4634-bd6a-c69210cb56ab None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:50.665040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:50.666076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.666619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.673143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.690450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd83cda-d059-481f-a29b-d8e7db7aa4ca None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:50.694902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-5339330e-2655-4e1d-8bd8-0c6827467d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:37:50.701523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:37:50.702397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:50.703173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:37:50.705416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 16/30] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:37:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:37:50.707974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7ec2-c76d-4ea6-ad8d-668ef01de696 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.708472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7ec2-c76d-4ea6-ad8d-668ef01de696 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.708971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7ec2-c76d-4ea6-ad8d-668ef01de696 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.711805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 17/31] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:50.717093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd7ec2-c76d-4ea6-ad8d-668ef01de696 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.760915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:50.761970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.762480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.762967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.764142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:50.765995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:50.814918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:50.865144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd7ec2-c76d-4ea6-ad8d-668ef01de696 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:50.902949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:50.903619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.903910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.904190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.905001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:50.905312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:50.906718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc00d8b-43e2-4ff7-9956-948a4a50c388 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:50.915602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 18/32] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:37:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:50.916630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:50.921612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:50.921982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:50.927929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 15/33] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:37:50 2022] POST /identity/v3/projects => generated 301 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:37:50.930864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:50.944336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:50.946791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:50.956065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.042738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 19/34] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:51.043383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:51.069304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d9e1f-e9e8-4ad7-9a92-f9f0b0288fbb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.141832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.154021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.154659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.154937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.155207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.160636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.160636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4552a-7e6d-4f36-9ee6-30d4e9de6fe5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.198919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 16/35] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Sep 20 04:37:50 2022] POST /identity/v3/roles => generated 225 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:37:51.252415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b868ed-1e66-47b7-ab4f-d84e8188cf95 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.296850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.297871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.298357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.298854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.300350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.305615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.307785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 20/36] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:37:51 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:51.314007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:51.314621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdd05-a5d4-4490-be87-5de6f340f134 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.314910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdd05-a5d4-4490-be87-5de6f340f134 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.315203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdd05-a5d4-4490-be87-5de6f340f134 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.318497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 17/37] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:51.328664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.328664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.328664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.348811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG passlib.handlers.bcrypt [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87797) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 20 04:37:51.349195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG passlib.handlers.bcrypt [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87797) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 20 04:37:51.406223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:51.407941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0863931b-d562-4af5-8e1a-d2f224cfac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:51.500718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016f7c4-b07e-40c6-9739-2091b0887544 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.521303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.531647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 18/38] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:51.533260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02159-1d03-4aec-ada0-256a31bbf337 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.533555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02159-1d03-4aec-ada0-256a31bbf337 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.533826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02159-1d03-4aec-ada0-256a31bbf337 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.535219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 19/39] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:51.535777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d02159-1d03-4aec-ada0-256a31bbf337 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:51.547145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.549466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.550334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.550832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.551087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.552794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.555884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227cb19-0e2c-41c6-b58e-147310e95447 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.557538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 21/40] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:51.558678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.558966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.559246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.578460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:51.579730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:51.654834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 22/41] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:51.655878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:51.673104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d02159-1d03-4aec-ada0-256a31bbf337 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.713022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.713639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.713918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.714188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.715139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.717276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.740771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 20/42] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Sep 20 04:37:51 2022] POST /identity/v3/roles => generated 219 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:37:51.785300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a62eb-5097-42f1-9b65-8cfcaf067155 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.808411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:51.808587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.839497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.840460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.840686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.840950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] PATH_INFO: `/v3/domains/default/users/095a92acb2e04f44b21db4de77e2a16a/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.842967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=095a92acb2e04f44b21db4de77e2a16a, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.844959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.889488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:51.927458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f408bf8-832c-43ca-9fbf-216a8e798475 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:51.967483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:51.968125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:51.968403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:51.968718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:51.970205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:51.971701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:51.973699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 21/43] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:51.974243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:52.080788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ec905-77ae-4e2c-a355-229bf8e13165 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:52.083175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 23/44] 10.222.0.27 () {58 vars in 1361 bytes} [Tue Sep 20 04:37:51 2022] PUT /identity/v3/domains/default/users/095a92acb2e04f44b21db4de77e2a16a/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:37:52.096808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:52.118669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:52.119300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:52.119580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:52.120025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:52.121119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:52.122832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94876b45-2295-4995-a2da-811aee2edd7c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:52.127841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:52.146198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 22/45] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Sep 20 04:37:51 2022] POST /identity/v3/roles => generated 223 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:37:52.211322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661b740-9300-40a3-8e56-dc91a5d8b295 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:52.252469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:52.253143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:52.253428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:52.253767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:52.254451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:52.256200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6cb2a-8f18-49ae-a97b-f0edcf1a0dc4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:52.295659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:37:52 2022] POST /identity/v3/projects => generated 312 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:37:54.242018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71b7c-ab02-4ede-9e97-2328dbcc65d4 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.242548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71b7c-ab02-4ede-9e97-2328dbcc65d4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.242548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71b7c-ab02-4ede-9e97-2328dbcc65d4 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.244040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:54.249195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546abd-4dc4-4005-be2a-08e36390f3ad None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.249604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546abd-4dc4-4005-be2a-08e36390f3ad None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.249751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546abd-4dc4-4005-be2a-08e36390f3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.268145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f1546abd-4dc4-4005-be2a-08e36390f3ad None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:54.269663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f1546abd-4dc4-4005-be2a-08e36390f3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:54.290090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:54.337784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:37:54.459182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9ee11-c341-4ac5-a3cb-a3bc2542ec01 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.459392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9ee11-c341-4ac5-a3cb-a3bc2542ec01 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.459673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9ee11-c341-4ac5-a3cb-a3bc2542ec01 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.461088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 24/49] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:54.466334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.466630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.466908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.485102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:54.486285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:54.551860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 26/50] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:54.608619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9ee11-c341-4ac5-a3cb-a3bc2542ec01 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:54.709849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9ee11-c341-4ac5-a3cb-a3bc2542ec01 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:54.742692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:54.743313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.743594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.743868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.755055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:37:54.760948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:37:54.761827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:37:54.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 25/51] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:37:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:37:54.767749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:54.821312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:54.852760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:54.867908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4c81-ce7f-43db-a0ac-f80db07310cf None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:54.877512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:54.901254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:54.901865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.902149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.902422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.903090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:54.903399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:54.904820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:54.912729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 27/52] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:37:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:54.918945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:54.922956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258bc01-d43d-4c64-b522-78b517697d88 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:54.955742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:54.956364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:54.956668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:54.956950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:54.958071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:54.959531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:54.962054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 26/53] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:54.967625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:55.022408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed0d-2b84-4305-a724-c1fe2cbce641 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:55.055163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:55.055789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:55.056068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:55.056339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:55.057498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:55.058938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:55.060649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 28/54] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:37:54 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:55.066078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:55.072237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f115-aa71-4ad1-a6ba-854766b5efbe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:55.084766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:55.095468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:55.107488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:55.108104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:55.108393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:55.108705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:55.109381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:55.110863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:55.139172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 27/55] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Sep 20 04:37:54 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:37:55.170976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0405183-1790-4645-b9fa-9a9b5830e016 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:55.207488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:55.208142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:55.208429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:55.208740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:55.209939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:55.212122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903ad75-24fa-4134-8f31-e416a61fe842 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:55.213738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 29/56] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:37:55.219790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:55.337381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54bb43-298b-4be7-827b-bc3197f34611 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:55.370824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:55.371433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:55.371716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:55.371991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:55.372738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.domain.id=default, user.id=095a92acb2e04f44b21db4de77e2a16a) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:37:55.373053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:55.374500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9b350-cdd8-43d3-9d92-2a810ad35dd4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:55.383713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 28/57] 10.222.0.27 () {56 vars in 1322 bytes} [Tue Sep 20 04:37:55 2022] GET /identity/v3/role_assignments?user.id=095a92acb2e04f44b21db4de77e2a16a&scope.domain.id=default&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 530 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:57.279918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e24e4-5b56-47d6-b5ee-3f3a395f070f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:57.280415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e24e4-5b56-47d6-b5ee-3f3a395f070f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:57.280518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e24e4-5b56-47d6-b5ee-3f3a395f070f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:57.281829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 30/58] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:37:57.286852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:57.287268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:57.287416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:57.306534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:37:57.307725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:37:57.341251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:57.477185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 29/59] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:37:57.746003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e24e4-5b56-47d6-b5ee-3f3a395f070f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:57.827768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:37:57.843901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e24e4-5b56-47d6-b5ee-3f3a395f070f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:57.876041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:57.876685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:57.876967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:57.877239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:57.878328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:57.879778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314edcc-4aca-4469-a818-aa49ac6f0774 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:57.881347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 31/60] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:37:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:37:57.887285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:37:57.982058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9db35a-f58d-4b83-a90c-542bbd006919 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:37:58.014260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:37:58.014799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:37:58.015079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:37:58.015353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:37:58.016002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:37:58.017510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f71db6-4b6f-4746-a162-4a78e328dd0c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:37:58.042911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 30/61] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:37:57 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:01.255479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba7ff-9373-4e5f-98a2-5d4b31bf5b11 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.256482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba7ff-9373-4e5f-98a2-5d4b31bf5b11 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.256984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba7ff-9373-4e5f-98a2-5d4b31bf5b11 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.259226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 32/62] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:01.265341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fdd0-5e9a-447d-9f03-377d8d8cda2f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.266271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fdd0-5e9a-447d-9f03-377d8d8cda2f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.266754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68fdd0-5e9a-447d-9f03-377d8d8cda2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.289959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4d68fdd0-5e9a-447d-9f03-377d8d8cda2f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:01.295204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4d68fdd0-5e9a-447d-9f03-377d8d8cda2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:01.320051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:01.390153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 31/63] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:01.480704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f78625-c6cf-4d35-b95b-4f0335a89e8a None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.480987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f78625-c6cf-4d35-b95b-4f0335a89e8a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.481178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f78625-c6cf-4d35-b95b-4f0335a89e8a None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.482668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 33/64] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:01.494198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48464ea5-a9e6-4f6d-9e7d-5e5757971a62 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.494488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48464ea5-a9e6-4f6d-9e7d-5e5757971a62 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.494766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48464ea5-a9e6-4f6d-9e7d-5e5757971a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.517641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-48464ea5-a9e6-4f6d-9e7d-5e5757971a62 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:01.519175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-48464ea5-a9e6-4f6d-9e7d-5e5757971a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:01.610955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 32/65] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:01.699463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f78625-c6cf-4d35-b95b-4f0335a89e8a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:01.799580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76e4e-5d63-4268-9dc7-0f1329999abe None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.799904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76e4e-5d63-4268-9dc7-0f1329999abe None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.800138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76e4e-5d63-4268-9dc7-0f1329999abe None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.801645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 33/66] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:01.811436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b606-1947-4ad3-9479-12cc1d6307c3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.811718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b606-1947-4ad3-9479-12cc1d6307c3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.811994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b606-1947-4ad3-9479-12cc1d6307c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.839627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ce36b606-1947-4ad3-9479-12cc1d6307c3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:01.839810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f78625-c6cf-4d35-b95b-4f0335a89e8a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:01.841288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ce36b606-1947-4ad3-9479-12cc1d6307c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:01.879919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:01.880714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.881004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.881493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.897365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-633fd74b-3ee9-4927-93c5-f99deb06c2b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:01.905314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:01.906282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:01.907071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 34/67] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:01.907823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6764-4406-44d1-b27c-db889030cbc7 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.908337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6764-4406-44d1-b27c-db889030cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.908847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e6764-4406-44d1-b27c-db889030cbc7 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.910853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 35/68] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:01.914450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e6764-4406-44d1-b27c-db889030cbc7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:01.925792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 34/69] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:01.934739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:01.935228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:01.935892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:01.945016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:01.974916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:01.977299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:01.989746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.043647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e6764-4406-44d1-b27c-db889030cbc7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.057477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 35/70] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:02.058153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.090212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.090822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.091156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.091446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.092501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:02.092850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.094509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a2fee-5acd-44d4-8f7f-0094d6ac367e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.105595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:02.109843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379d5e0-32d6-4772-959d-84bb8d67c0c0 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.110347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379d5e0-32d6-4772-959d-84bb8d67c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.111514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379d5e0-32d6-4772-959d-84bb8d67c0c0 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.112793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 37/72] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:02.113497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379d5e0-32d6-4772-959d-84bb8d67c0c0 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.179818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fdcd73-4588-4f19-89d4-33b435fdaf1d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.216020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.216915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.217373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.217785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.225519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.230408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-aa4fb778-fc18-4d04-83e7-e94fbd11a2b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:02.235748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:02.236631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:02.237484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 36/73] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:02.238735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.238839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.239106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.251046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.256848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:02.258100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:02.260141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379d5e0-32d6-4772-959d-84bb8d67c0c0 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.296822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.297768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.298113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.298452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.311987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.313409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.315212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 38/74] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:02.320214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.328851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 37/75] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:02.330264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.421121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3cc35-2088-4689-a653-9c1742077206 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.437351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d654a2-9c0f-4f94-8226-2e698187c75f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.450948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.457657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.458367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.458659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.458934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.469578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:02.473095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.473891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.474355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:02.474473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:02.475251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:02.476025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 39/76] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:02.476708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.477451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.478129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:02.478436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.479816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.484689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.499276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 38/77] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:02.500145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.582280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b5b0-ecc4-4c1b-a25d-a96bfc4caba8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.603339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc47ddf-59aa-4d8f-96ad-98abf9c37826 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.616295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.616952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.617243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.617514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.637819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.638443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.638735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.639002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.649571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:02.654476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:02.655357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:02.656144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 39/78] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:02.656832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.675809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.685493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.687005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.714364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.754465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c269a-f2ab-4994-a5cc-91c24da348a3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.785872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 40/79] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:02.786803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.793882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.794492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.794775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.795055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.795714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:02.796013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.797427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.806372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 40/80] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:02.807146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.878703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.887461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39557c36-cddf-4ed2-9cf9-6d3c9ee12399 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.905813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19226e-7dea-435c-8152-49d7cec64d7d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:02.922145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.922777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.923059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.923337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.924447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.925926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.926335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:02.927696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 41/81] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:02.928328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:02.941211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:02.941845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:02.942126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:02.942407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:02.943515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:02.945084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:02.946786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 41/82] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:02.947404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.029900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ab318-f295-4660-a5d3-3e2a34127ad3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.049444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a8e10-2380-47bc-9a25-2733b2dc3fc1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.062185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.062793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.063069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.063337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.063991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:03.064290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.065709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.073596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 42/83] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.074672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.086599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.087227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.087508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.087781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.088925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.090346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.091940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 42/84] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:02 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.092718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.099785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.154511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.176803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e6adf-e486-40f5-a608-2d0453cddffd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.198536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c95469-c50d-4a73-96f7-794a8697b6f7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.209126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.209753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.210036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.210311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.212053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.213543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.215110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 43/85] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.215933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.233170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.233777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.234062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.234337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.299994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.301486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.303465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.314254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c9dff-f323-4816-a55d-856943c23baf None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.347842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.348450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.348765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.349040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.349692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.351648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.357556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.377473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 44/86] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:03 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:03.385634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.404999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 43/87] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:03.405621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.510577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc34d2-6884-4f91-a69d-99e9d299b3ff None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.528492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a187-2835-4431-a176-b4be46226b04 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.547518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.548161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.548443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.548741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.550136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.551582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.553281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 45/88] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.554066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.567944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.568558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.568867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.569140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.569897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:03.570810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.570915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.572072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.581292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 44/89] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=3fbe9323075347109c2f57834701d0de => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.582208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.607975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.691996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef1e17-b3c6-4199-8e24-9b709a605518 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.713033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b450f-2409-4417-9f96-e73f8bf9686b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.753312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.753945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.754225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.754499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.766217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:03.767049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:03.767424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:03.768180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:03.769853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.771427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.773978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 45/90] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:03.774959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:03.835900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:03.838222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:03.854092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.910907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:03.944549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cfde2-06f2-4fb5-bae9-0798a4134fd4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:03.953680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 46/91] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:03.954737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:04.026369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.027139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.027524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.027893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.116053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709504-d570-4389-a1f7-a24a4244eb47 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:04.140438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.142140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:04.142444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.219904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.220613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.221001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.221373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.222224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=095a92acb2e04f44b21db4de77e2a16a) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:04.222631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.224193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.249468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:04.250301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 47/92] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/role_assignments?user.id=095a92acb2e04f44b21db4de77e2a16a&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:04.251335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:04.262530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 46/93] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:04.277712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:04.437447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ed412-74d8-4221-a440-e25d57f8a1c0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:04.478128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.478728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.479013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.479295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.480546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.483952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.483952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 47/94] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:04 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:04.490194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:04.505649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91bcfd7-78c4-42d9-90cd-cc39d902895e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:04.581418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.582198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.582488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.582764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] PATH_INFO: `/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.584006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] RBAC: Authorizing `identity:get_project(project_id=1df8bbc2b1774ae7bed5ad4bab7d26d7)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.585539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.587341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 48/95] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:04 2022] GET /identity/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7 => generated 312 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:04.593236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:04.609980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:04.664976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:04.706302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1cfb-d8f8-4e1f-ad60-49dbd6cc3e81 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:04.768643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882b6a9-e330-494e-8733-3a353d1ca37c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:04.781691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.782317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.782593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.782862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.783595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:04.783904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.785369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a4ac4-07fc-4d28-8e1d-488884c2a6b1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.796996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 48/96] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:04 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=d53450bb086544ce968b603c0997c5b9 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:04.834996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:04.835690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:04.835980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:04.836265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:04.837047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=1df8bbc2b1774ae7bed5ad4bab7d26d7, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:04.837369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:04.838842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762567d-1211-46be-9ca1-76c532ce9edf None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:04.844621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 49/97] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:04 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=1df8bbc2b1774ae7bed5ad4bab7d26d7&role.id=3fbe9323075347109c2f57834701d0de => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:06.270619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac344de4-1baa-4a05-be28-a15d242320b9 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:06.271182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac344de4-1baa-4a05-be28-a15d242320b9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:06.271182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac344de4-1baa-4a05-be28-a15d242320b9 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:06.272506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 49/98] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:06.278263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fd12c-c1c3-4eb1-b849-80e6f9bd28c3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:06.278667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fd12c-c1c3-4eb1-b849-80e6f9bd28c3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:06.278856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fd12c-c1c3-4eb1-b849-80e6f9bd28c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:06.305481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4c6fd12c-c1c3-4eb1-b849-80e6f9bd28c3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:06.306757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4c6fd12c-c1c3-4eb1-b849-80e6f9bd28c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:06.398342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 50/99] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:06.594567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf856c-6c1b-4f4d-9575-609ecbeab6f7 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:06.594941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf856c-6c1b-4f4d-9575-609ecbeab6f7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:06.595117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf856c-6c1b-4f4d-9575-609ecbeab6f7 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:06.596737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 50/100] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:06.602408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:06.602705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:06.602984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:06.629529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:06.630720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:06.672228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:06.714259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 51/101] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:06.930835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf856c-6c1b-4f4d-9575-609ecbeab6f7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.059404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf856c-6c1b-4f4d-9575-609ecbeab6f7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.088102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:07.107705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.108377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.108679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.108956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.130330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-93569c7a-a6f9-4eb2-91d1-5c1962fa0c67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:07.136913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:07.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:07.138555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 51/102] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:07.142439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.215704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f2-3746-4468-86e7-c87d058bf974 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.215994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f2-3746-4468-86e7-c87d058bf974 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.216173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbd3f2-3746-4468-86e7-c87d058bf974 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.217738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 52/103] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:07.222347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.222636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.222910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.243081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:07.244289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:07.323956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d27ce6-8f4f-46e7-8185-0776f07baa66 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.325749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 53/104] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:07.326749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.388144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.388797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.389080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.389354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.390037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:07.390341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:07.397012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:07.414770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 52/105] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:07.420326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.485085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:07.541006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284e4ea-e935-486a-917c-fe725b62f954 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.568178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:07.615120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.616132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.616632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.617480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.620238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:07.621545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da217bf-9f84-41b6-8f86-72d1820c1f5b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.623213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:07.626490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 54/106] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:07.632523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.719391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.720448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.721195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.723626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] PATH_INFO: `/v3/users/fb2828e1808a467b882eaec92d295102` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.744094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=fb2828e1808a467b882eaec92d295102)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:07.747634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:07.751279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 53/107] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/users/fb2828e1808a467b882eaec92d295102 => generated 289 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:07.754361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.837050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be645f1f-4e9a-4909-b5b2-896a77403036 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.875420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.876026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.876322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.876578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.877275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:07.878754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:07.888681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:07.901848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586baa8c-d459-48c2-86db-92d92a0bb6c1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:07.907194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 55/108] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:38:07 2022] POST /identity/v3/projects => generated 302 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:07.908182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:07.946469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:07.947255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.947708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.948046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.953441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:07.962586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-3bad5151-403e-434e-819d-0d75c729b43c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:07.968573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:07.969626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:07.970395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 54/109] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:07.974135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a89fd-92b8-4ed2-9e64-aec870ca3422 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.974135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a89fd-92b8-4ed2-9e64-aec870ca3422 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.974135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a89fd-92b8-4ed2-9e64-aec870ca3422 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.974135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 55/110] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:07.975104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b1ce3-466a-406a-a018-b9edee3b5db8 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:07.975425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b1ce3-466a-406a-a018-b9edee3b5db8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:07.975770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b1ce3-466a-406a-a018-b9edee3b5db8 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:07.977836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 56/111] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:07.978484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b1ce3-466a-406a-a018-b9edee3b5db8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.044699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4719-2962-44c4-9581-a9071b78dde9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.090473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.091581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.092262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.092542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.098520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b1ce3-466a-406a-a018-b9edee3b5db8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.139736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.140427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.140741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.141016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.141690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:08.141993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:08.143382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5cfa0-fcaa-4de0-a68c-b080a5e4165b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:08.154999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 57/112] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:08.156215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7c03-44b2-4cbb-bf68-b734666c2d20 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.156488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7c03-44b2-4cbb-bf68-b734666c2d20 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.157056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7c03-44b2-4cbb-bf68-b734666c2d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.175251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-457b7c03-44b2-4cbb-bf68-b734666c2d20 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:08.176427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-457b7c03-44b2-4cbb-bf68-b734666c2d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:08.184380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.188392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:08.189965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6674f-0cef-4a2b-a0e0-8413148233cf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:08.221717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.248719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 58/113] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:08.250845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52b42-7976-44ff-8a6c-2eb58c2803f8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.251492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52b42-7976-44ff-8a6c-2eb58c2803f8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.252277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b52b42-7976-44ff-8a6c-2eb58c2803f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.275331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-24b52b42-7976-44ff-8a6c-2eb58c2803f8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:08.277380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-24b52b42-7976-44ff-8a6c-2eb58c2803f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:08.296412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 56/114] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:08.297548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d76c-6de8-404d-9d58-1a9e268c365e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.297913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d76c-6de8-404d-9d58-1a9e268c365e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.298262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1d76c-6de8-404d-9d58-1a9e268c365e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.299690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 57/115] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:08.300354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d76c-6de8-404d-9d58-1a9e268c365e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.374441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 59/116] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:08.376380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.376853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.377357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.407615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:08.409796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:08.462428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1d76c-6de8-404d-9d58-1a9e268c365e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.522889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.535669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.536890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.537882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.539802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 60/117] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:08.540269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.541817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.542841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:08.545546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:08.547334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 58/118] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:08.548891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.555110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.687496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd28d49-ea93-4c5b-86a1-4b0530c49f12 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.729187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.730128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.730672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.731116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.732527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:08.732728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3874d1-5ff2-4e1f-a452-d22e4b7c382e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.734381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:08.736196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 59/119] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:08.739686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.774925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.775559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.775837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.776108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.777820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:08.779264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:08.780906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 61/120] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:08.781675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.828581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.852278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:08.867228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e10c5-c6a7-455a-894d-07faa209a8a2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.885350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbcc5a-7c95-4d11-8e52-3c81d1995f0b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:08.907117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.907758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.908040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.908321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.918897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:08.922208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:08.922832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:08.923112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:08.923384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:08.924161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:08.925059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 60/121] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:08.930068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:08.939843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:08.945862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:08.946726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 62/122] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:08.947745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.032582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c9ac5-6c2c-45b1-bbb0-8b9b4e1047e8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.060396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6a114-b430-4a89-886e-e5dc1fd58581 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.071577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.084518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.085561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.086015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.086468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.089638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, user_id=4359f48b5edc49bc97c3e969f9e4a94e, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.092146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.104299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.120631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 61/123] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:08 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:09.120631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.122113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.122757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.123040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.123372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.124446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=fb2828e1808a467b882eaec92d295102) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:09.124793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.130636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.139012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 63/124] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:08 2022] GET /identity/v3/role_assignments?user.id=fb2828e1808a467b882eaec92d295102&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=7bf924eb503449ccaddc82aad1e450af => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:09.140068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.330851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a28b0-28f8-4765-9834-adc577261377 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.345838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3006a4-3307-4525-b1a1-dc4054670632 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.383367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.384021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.384385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.384724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.385533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:09.385838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.387291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.396772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.397419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.398032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.398618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 64/125] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:09.398693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.399442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.412218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:09.419586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:09.420722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 62/126] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:09.421691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.476972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.501956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.554016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb7dd3-d1b5-4293-9ab0-34227f71cf21 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.596138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7789350-dbda-4a30-9783-00bd2fb3c91e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.631846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.633161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.633453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.633730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.634522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:09.634833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.635235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.636024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.636251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.636590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.637052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.645362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.646981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.650378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 63/127] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:09.651320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.655664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 65/128] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:09.657550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.808821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630f75a-21a4-4d8e-ac0e-c65471823289 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.826355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.853580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.854419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.855785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.856197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.857058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:09.857625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.859176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.869228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 64/129] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:09.870929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:09.904325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb419b-f918-4f2c-9728-ddc6d1b2d65f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.952873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:09.979417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219361a-8b81-4988-b796-51831e68cca6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:09.992075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:09.992760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:09.993050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:09.993330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:09.994477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:09.995989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:09.997612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 66/130] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:10.003012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.042513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.043449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.043910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.044345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.046415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.048015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.050669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 65/131] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:10.053663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.136219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:10.177235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abedc6d-db41-45bc-8c3e-a63f41501614 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.216628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.216628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.216628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.216628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.217068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.217743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.219476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 67/132] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:09 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:10.220345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.243564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d8785-11d4-4a72-b828-5499b0a6512d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.298341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:10.302143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.303199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.305782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.306190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.309203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, user_id=4359f48b5edc49bc97c3e969f9e4a94e, role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.311414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.344323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 66/133] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:10 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/d53450bb086544ce968b603c0997c5b9 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:10.345376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.394054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa634f-c71a-4443-8449-ac0b183108ab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.475586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.476900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.477191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.477470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.481394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.482901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f5671-24a3-4687-911b-ab70bee276db None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.484676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 68/134] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:10 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:10.485886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83a30-1b98-4ee4-9219-473e35027f40 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.486175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83a30-1b98-4ee4-9219-473e35027f40 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.486451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83a30-1b98-4ee4-9219-473e35027f40 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.487663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 69/135] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:10.493042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.493444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.493719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.494310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab87c82-2141-47e4-a3f7-119aeac99aeb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.497261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:10.532713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:10.534063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:10.545350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.546788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.547290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.547765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] PATH_INFO: `/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.549975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] RBAC: Authorizing `identity:get_project(project_id=1df8bbc2b1774ae7bed5ad4bab7d26d7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.551828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.553780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 67/136] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:10 2022] GET /identity/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:10.554818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.634148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 70/137] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:10.634987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:10.686865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debb7ac-b8f9-407c-bc70-49b325b8d9aa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.708908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:10.753731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.754739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.755028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.755306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/095a92acb2e04f44b21db4de77e2a16a/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.757796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, user_id=095a92acb2e04f44b21db4de77e2a16a, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.759492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.767918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f03b3-6a6f-47d7-8233-a5fe3ce16a47 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:10.792476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 68/138] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:10 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/095a92acb2e04f44b21db4de77e2a16a/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:10.823501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:10.824180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:10.824463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:10.824773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] PATH_INFO: `/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:10.827681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] RBAC: Authorizing `identity:create_grant(project_id=1df8bbc2b1774ae7bed5ad4bab7d26d7, user_id=4359f48b5edc49bc97c3e969f9e4a94e, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:10.829254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:10.860227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 71/139] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:10 2022] PUT /identity/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7/users/4359f48b5edc49bc97c3e969f9e4a94e/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:11.117567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:11.167633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:11.250795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68379-0cd5-44cb-b199-d23f923186fe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:11.298134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:11.298773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:11.299052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:11.299326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:11.300443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:11.301903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b590ac2-1ba7-4f55-b50c-b5cdff123df0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:11.303779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 69/140] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:11.309375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:11.431568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882a35b-9e8b-4609-b044-c5b14b7a9a55 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:11.453625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:11.473390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:11.474025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:11.474311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:11.474589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:11.475510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:11.477576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681c488e-1e9e-41bc-8026-375be7fd40e7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:11.515841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 72/141] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:11 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:11.963626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbff3-6996-4a80-931a-6aeb6ee61844 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:11.964134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbff3-6996-4a80-931a-6aeb6ee61844 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:11.964238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fbff3-6996-4a80-931a-6aeb6ee61844 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:11.965861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 70/142] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:11.980550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:11.980885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:11.981170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.009857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:12.011108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:12.017459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1e03c-27f4-430e-bfc4-b31b568d60c3 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:12.018081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1e03c-27f4-430e-bfc4-b31b568d60c3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:12.018556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1e03c-27f4-430e-bfc4-b31b568d60c3 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.020155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 71/143] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:12.027432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:12.028037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:12.028508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.047359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:12.049026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:12.134264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 73/144] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:12.146250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 72/145] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:12.428073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:12.532560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:12.536944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fe8f8-e670-444e-97c7-d27093228d10 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:12.555704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:12.590169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:12.590815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:12.591097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:12.591369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.602540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:12.607359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:12.608211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:12.609398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 74/146] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:12.612580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:12.614225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:12.754809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551bbff2-f63e-4755-9f62-681e100bc66f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:12.795086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:12.795599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ae7c4-17ce-443f-a8d1-989e0ca2351c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:12.795766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:12.796034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:12.796318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.797289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:12.797594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:12.799946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:12.810044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 75/147] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:12.816348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:12.842961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:12.868556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:12.869230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:12.869508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:12.869777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:12.897066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:12.902245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:12.903116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:12.903905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 73/148] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:12.910777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:12.970647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e730397-6fac-4785-ada6-5ea6aa27855e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.020631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.032497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e4ac3-5c02-4f8a-91ed-5847a507e51b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.040541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.041217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.041502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.041778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.042947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.044411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.046322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 76/149] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:12 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:13.059202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.085070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.085773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.086047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.086315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.087307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:13.087612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.090833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.111094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 74/150] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:13.122445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.243759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.260866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9dea5-98e5-4361-aca0-605bd5861b94 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.321098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.321895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.322179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.322510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.347059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311827b-2687-45df-b426-95a0d6cee0e7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.422964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.423166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.423431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.423725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] PATH_INFO: `/v3/users/fb2828e1808a467b882eaec92d295102` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.429393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] RBAC: Authorizing `identity:get_user(user_id=fb2828e1808a467b882eaec92d295102)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.430894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.432753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 75/151] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:13 2022] GET /identity/v3/users/fb2828e1808a467b882eaec92d295102 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:13.433546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.434902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.457914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.498555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.562746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.579620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 77/152] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:13.586253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.613809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e53a7-d68e-4d30-a06a-34a4484bcbc1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.670288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.670969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.671259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.671521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.677367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.678859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.680789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 76/153] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:13 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:13.687610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.778456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55066d08-0e90-4986-a810-896b36c0bef2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.796979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.819147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738255-f61b-40c2-8de3-95c56bea10e8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:13.831864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.832965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.833484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.834045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.835436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.837332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.845071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 78/154] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:13 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:13.846362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:13.864628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:13.864628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.864628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.864628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/fb2828e1808a467b882eaec92d295102/roles/7bf924eb503449ccaddc82aad1e450af` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.865019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, user_id=fb2828e1808a467b882eaec92d295102, role_id=7bf924eb503449ccaddc82aad1e450af)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:13.865019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48d6c-7fa2-4963-a517-bbbd72ffb2fc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:13.878709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 77/155] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:13 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/users/fb2828e1808a467b882eaec92d295102/roles/7bf924eb503449ccaddc82aad1e450af => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:38:13.934707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:13.980967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ae836-fecd-4933-bc76-65f9c986abbe None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:13.981312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ae836-fecd-4933-bc76-65f9c986abbe None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:13.981515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ae836-fecd-4933-bc76-65f9c986abbe None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:13.983031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 78/156] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:13.988558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6c8f5-58c3-4aac-912b-aa7bf1e32009 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:14.006257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd331b-c3e9-47f5-8e20-c350c695699a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.006556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd331b-c3e9-47f5-8e20-c350c695699a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.006806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd331b-c3e9-47f5-8e20-c350c695699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.026310 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:14.027346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.027784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.028250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.029158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:14.029624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:14.031909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e8a32-aecf-4765-94ae-9260f071c4d9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:14.039670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 79/157] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:13 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=3fbe9323075347109c2f57834701d0de => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:14.049896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d2dd331b-c3e9-47f5-8e20-c350c695699a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:14.051125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d2dd331b-c3e9-47f5-8e20-c350c695699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:14.182408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 79/158] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:14.652518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db30d0-bc16-4835-8dcb-03491d45793c None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.653381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db30d0-bc16-4835-8dcb-03491d45793c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.653797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db30d0-bc16-4835-8dcb-03491d45793c None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.655374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 80/159] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:14.665628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf23c-73d0-493e-b81e-55fc091f1e25 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.666169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf23c-73d0-493e-b81e-55fc091f1e25 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.666581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bf23c-73d0-493e-b81e-55fc091f1e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.686135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-818bf23c-73d0-493e-b81e-55fc091f1e25 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:14.687603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-818bf23c-73d0-493e-b81e-55fc091f1e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:14.754039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db30d0-bc16-4835-8dcb-03491d45793c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:14.794550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 80/160] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:14.911591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db30d0-bc16-4835-8dcb-03491d45793c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:14.937313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:14.957163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:14.957798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.958077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.958348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.980552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:14.985440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:14.986369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:14.987179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 81/161] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:14.987843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31489-3beb-446f-b383-3f7b1f43df6e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.987843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31489-3beb-446f-b383-3f7b1f43df6e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.987843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31489-3beb-446f-b383-3f7b1f43df6e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:14.988950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 81/162] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:14.993337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:14.994706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:14.994986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:14.995259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.036638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:15.037840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:15.040733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:15.153044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd2ca1-efd8-4c5d-bd49-d0897d7ef67b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:15.215742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:15.216749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:15.217256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:15.217790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.218668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:15.219177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:15.220889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:15.231920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 82/163] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:15.237304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:15.238447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 82/164] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:15.302260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:15.379144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:15.385371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240ea62-d285-4767-970c-3c0117994c51 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:15.439013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:15.439657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:15.439937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:15.440211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.445390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:15.447006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:15.448930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 83/165] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:15 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:15.454857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:15.617686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5104d-abf6-4f05-85ca-f338beb490f5 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:15.628245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5171339-463a-41dd-aee0-cc71ba6da0a2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:15.643161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:15.663263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:15.665262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:15.665542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:15.665814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.680187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:15.692533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:15.700520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:15.701667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:15.702752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:15.704263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 83/166] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:38:15.710784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:15.711396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:15.711678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:15.711953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.714790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:15.884189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdf778-9462-4b12-805e-aa36661fb71f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:15.902689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:15.904166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:15.928220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:15.928902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:15.929706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:15.929976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:15.946166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:15.951018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:15.951939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:15.952871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 84/167] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:15.956823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.049079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.110619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 84/168] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 20 04:38:16.111500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.139279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.143784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293f2c9-e991-4d86-b67a-9e23dbf552fa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.210338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.211020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.211322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.211571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.212235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:16.212537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.213958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.222558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 85/169] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.223451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.326632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76d9e-a962-4a6b-980f-2cab23f97e53 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.377966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d522262-d95f-4cc7-bf38-6577dfb1ab2b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.418141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.418678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.418956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.419225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.419901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:16.420204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.420621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.422601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.439872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 85/170] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.440797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.450279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.450870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.451151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.451421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.457314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.458776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.461283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 86/171] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.462196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.619306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.675577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bdfe2-888e-43c2-84ac-6f2f121c35d1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.706016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b355-f37a-41d8-a8dd-4ab08eecc46e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.734780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.736137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.736661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.737164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] PATH_INFO: `/v3/users/095a92acb2e04f44b21db4de77e2a16a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.738520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] RBAC: Authorizing `identity:get_user(user_id=095a92acb2e04f44b21db4de77e2a16a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.740201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.742146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 86/172] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/users/095a92acb2e04f44b21db4de77e2a16a => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.743158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.773565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.774214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.774741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.775013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] PATH_INFO: `/v3/users/4359f48b5edc49bc97c3e969f9e4a94e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.776170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] RBAC: Authorizing `identity:get_user(user_id=4359f48b5edc49bc97c3e969f9e4a94e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.777634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.782260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 87/173] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/users/4359f48b5edc49bc97c3e969f9e4a94e => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.785161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.865835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.866240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ea9a-6816-4d8d-9ee1-2f939fdf4861 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.905176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.905786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.906140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.906336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.907053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:16.907359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:16.908837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd893537-d66b-4e4f-b392-1388dd4f5bc6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:16.915794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 87/174] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=d53450bb086544ce968b603c0997c5b9 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:16.917181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1686cd-ae14-4df8-85a0-8a1589aec221 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.917466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1686cd-ae14-4df8-85a0-8a1589aec221 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.917737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1686cd-ae14-4df8-85a0-8a1589aec221 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.923254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 88/175] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:16.924742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e2df6-0007-43e5-854f-d825b32a21ea None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.925027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e2df6-0007-43e5-854f-d825b32a21ea None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.925297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e2df6-0007-43e5-854f-d825b32a21ea None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:16.932169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 89/176] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:16.933074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e2df6-0007-43e5-854f-d825b32a21ea None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:16.936202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb846b-36d8-4fb3-94ed-a0ac6d436b3a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:16.938019 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:16.993057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:16.993699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:16.993979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:16.994252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.091333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e2df6-0007-43e5-854f-d825b32a21ea None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:17.100287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.108702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460107a8-76f2-4b2c-8dd4-7828f5e3fa12 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.138401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:17.139583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.140126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.140623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.190183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.231074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.232558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fef1a6-52c4-4474-98c4-ba30f6f00667 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.235837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.243801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 88/177] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:17.246026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405d9e7-1129-4a43-bbbf-b3446389e117 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.246475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405d9e7-1129-4a43-bbbf-b3446389e117 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.246890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405d9e7-1129-4a43-bbbf-b3446389e117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.267857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c405d9e7-1129-4a43-bbbf-b3446389e117 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:17.269326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c405d9e7-1129-4a43-bbbf-b3446389e117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:17.352635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 89/178] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:17.352635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.352635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.353002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.364755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 90/179] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:17.365804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e23d7-1a1e-4a72-a4d0-3f1399b338b9 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.366109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e23d7-1a1e-4a72-a4d0-3f1399b338b9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.368594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e23d7-1a1e-4a72-a4d0-3f1399b338b9 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.368594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 91/180] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:17.368594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e23d7-1a1e-4a72-a4d0-3f1399b338b9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:17.392308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:17.393525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:17.522766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 90/181] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:17.524958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:17.553128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.562853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e23d7-1a1e-4a72-a4d0-3f1399b338b9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:17.566379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.626318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:17.628636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.629356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.630506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.633139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.634991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48477f-9580-4f52-b7e6-df615fcd0d74 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.639043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 92/182] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:17 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:17.640395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dffec-7813-4436-b0ce-444200644f12 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.640930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dffec-7813-4436-b0ce-444200644f12 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.641413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dffec-7813-4436-b0ce-444200644f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.669775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-852dffec-7813-4436-b0ce-444200644f12 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:17.670958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-852dffec-7813-4436-b0ce-444200644f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:17.688478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a850b7-4228-4b96-a680-ca4b460c3ffb None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:17.727995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:17.728647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.728915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.729182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] PATH_INFO: `/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.730934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] RBAC: Authorizing `identity:get_project(project_id=1df8bbc2b1774ae7bed5ad4bab7d26d7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.732760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d7458-3c6f-476a-951f-6d0ed0cc3cb3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.734480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 91/183] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:17 2022] GET /identity/v3/projects/1df8bbc2b1774ae7bed5ad4bab7d26d7 => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:17.735572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca92b-3682-4d0c-bf5a-15eb7a5b9ff7 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.735846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca92b-3682-4d0c-bf5a-15eb7a5b9ff7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.736111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca92b-3682-4d0c-bf5a-15eb7a5b9ff7 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.737458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 92/184] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:17.737975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fca92b-3682-4d0c-bf5a-15eb7a5b9ff7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:17.751863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 93/185] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:17.752846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dffec-7813-4436-b0ce-444200644f12 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:17.856813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.866184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dffec-7813-4436-b0ce-444200644f12 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:17.868843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:17.874579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fca92b-3682-4d0c-bf5a-15eb7a5b9ff7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:17.905327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:17.906009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.906287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.906554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.907679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.909230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.910934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 94/186] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:17.911506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:17.911806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:17.912493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.912804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.913076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.914364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=095a92acb2e04f44b21db4de77e2a16a) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:17.914674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:17.916193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23d35-ff6d-4b05-9fc2-5d03869c5161 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:17.932674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 93/187] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:17 2022] GET /identity/v3/role_assignments?user.id=095a92acb2e04f44b21db4de77e2a16a&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:17.932674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:17.932674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:17.932674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:17.946661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:17.948079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:18.031121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 94/188] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:18.032072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.055069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f7fed-117a-4605-9d0c-7d6d7052f77c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.096714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.097445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.097828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.098105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.098851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=1df8bbc2b1774ae7bed5ad4bab7d26d7, user.id=4359f48b5edc49bc97c3e969f9e4a94e) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:18.099161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.100741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.107387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 95/189] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:17 2022] GET /identity/v3/role_assignments?user.id=4359f48b5edc49bc97c3e969f9e4a94e&scope.project.id=1df8bbc2b1774ae7bed5ad4bab7d26d7&role.id=3fbe9323075347109c2f57834701d0de => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:18.108205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.141818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.161246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.174753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95ccd-bf94-44b0-84a8-8fcea952e3f3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.225109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.226645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.227144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.227583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.240988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:18.246760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:18.247710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:18.248594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:18.251080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 95/190] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:18.252386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.271568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf79cf-c92a-42d1-83a7-064f99f2d5ea None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.320739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.321516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.321908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.322254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.323602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.325463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.356027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 96/191] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:18 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:18.356854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.380034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8364c-8185-4b25-bab1-0f82af9360d9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.420979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.421645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.422179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.422179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.433950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.440495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:18.447760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:18.448761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 96/192] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:18.449594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.484013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.496760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5da83-bc0d-4594-8228-c68f44976db8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.540317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.540964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.541242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.541515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.542187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:18.542832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.544627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.554570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 97/193] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:18.557457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.571979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d69be-0934-4ee1-866e-0f139bcb60c6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.615085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.615983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.616967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.617393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.627903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:18.635212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:18.636106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:18.637008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 97/194] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:18.637731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.676789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294b52c-f3d7-447c-8e88-cca3550acc40 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.708926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.724698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.725322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.725602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.726474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.727892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:18.727892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.728886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.738556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 98/195] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:18.739813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.756998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:18.766401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c4842-4706-43cb-bada-5469404e35d0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.810906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.811851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.812239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.812634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.830939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.832589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.839760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 98/196] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:18.839760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.883045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a775-6e48-465f-bddb-2dc354e631bb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.923426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:18.924114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:18.924497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:18.924895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:18.925664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:18.926089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:18.927726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:18.943344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 99/197] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:18.943344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:18.969043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff323492-92c3-42fa-a021-a535421f49f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:18.986803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.006731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.007419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.007700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.007973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.009287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.010796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.012493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 99/198] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:19.013277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.060973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.084634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a05a2-b672-48d6-91a3-aaa3587d0573 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.126932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b430-c5db-413b-b01a-093ac6d828b7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.137808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.138454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.138733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.139000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.169376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.170394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.170394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.170689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] PATH_INFO: `/v3/users/fb2828e1808a467b882eaec92d295102` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.171821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] RBAC: Authorizing `identity:get_user(user_id=fb2828e1808a467b882eaec92d295102)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.173590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.180190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 100/199] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/users/fb2828e1808a467b882eaec92d295102 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:19.180190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.242368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.250254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.252548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.296127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6d5b5-ff9a-4e1d-a9f5-e5cfd5a11cbb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.343141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.344134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.345121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.345601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.346876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.377032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 100/200] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:19.377032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.433844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.435384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.500145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57e4e-651e-46f1-86b8-129a09418fa6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.506399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.536632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.537399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.537770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.538134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.574177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 101/201] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:19.575308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.591090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.624068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.627321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.697907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba51a-2fdb-4caa-ba25-532cce1e6240 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.739756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 101/202] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:19.739756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.748685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.749550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.750266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.750878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.752290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.754671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.761863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 102/203] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:19.761863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.792878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.830065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:19.859112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac1308-8f81-4df0-a011-a9caa8ea4f13 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.895646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.896286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.896568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.896884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.897569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61674d92-ccf2-4736-af2a-d22fa6a45fb8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:19.898057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.899869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.901641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 102/204] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:19.902415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:19.938819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:19.939803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:19.940263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:19.940738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:19.942030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:19.943746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:19.945718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 103/205] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:19.952169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:20.026285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5076f-51a1-459d-a868-56dea7444aeb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:20.050193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:20.064233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa601-cf86-4736-a7ab-d5affb3fa7a2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:20.075167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:20.076594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.077147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.077587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.078725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=f714b6b7421c4d6db5f5f173dc172c0f) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:20.079221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:20.081526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:20.096639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 103/206] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/role_assignments?user.id=f714b6b7421c4d6db5f5f173dc172c0f&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:20.096639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:20.113675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:20.117883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:20.119858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.119858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.119858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.127348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=f714b6b7421c4d6db5f5f173dc172c0f) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:20.127348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:20.130755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404183e-9c96-4ad3-845c-3ea1acef6c7a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:20.144968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 104/207] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:19 2022] GET /identity/v3/role_assignments?user.id=f714b6b7421c4d6db5f5f173dc172c0f&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=d53450bb086544ce968b603c0997c5b9 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:20.244855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a361a5c-74eb-4486-867c-c2a59873a71e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:20.291542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:20.292187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.292470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.292772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.293537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, scope.project.id=e9483c5eaf14474197c855467cfe5a7d, user.id=fb2828e1808a467b882eaec92d295102) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:20.293847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:20.295419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70d770-db19-414f-bb47-80d2488c496e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:20.302998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 104/208] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:20 2022] GET /identity/v3/role_assignments?user.id=fb2828e1808a467b882eaec92d295102&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=7bf924eb503449ccaddc82aad1e450af => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:20.735937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a54a-0179-4998-9db9-b54829aaebcc None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.736422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a54a-0179-4998-9db9-b54829aaebcc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.736422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1a54a-0179-4998-9db9-b54829aaebcc None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.737900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 105/209] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:20.743377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd222-81a1-4225-a67e-4d798773263c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.743668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd222-81a1-4225-a67e-4d798773263c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.743944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfd222-81a1-4225-a67e-4d798773263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.765540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-68dfd222-81a1-4225-a67e-4d798773263c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:20.766743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-68dfd222-81a1-4225-a67e-4d798773263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:20.849512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 105/210] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:20.925252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098183-26fd-4046-a066-dd7911e80428 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.925556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098183-26fd-4046-a066-dd7911e80428 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.925788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098183-26fd-4046-a066-dd7911e80428 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.927404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 106/211] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:20.932357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:20.932911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:20.933384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:20.940136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:20.955775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:20.957287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:21.029698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 106/212] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:21.134869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01098183-26fd-4046-a066-dd7911e80428 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.247709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01098183-26fd-4046-a066-dd7911e80428 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.283189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.283749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.284023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.284292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.294608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:21.299898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:21.300808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:21.302238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.302310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 107/213] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:21.306745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.323157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:21.404673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d62f0-9284-462a-aa1f-7151e7cc5572 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.418449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222c7bc-fed5-4ac1-8876-a5ea6f817427 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.440796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:21.448290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.449293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.449835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.450281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.451553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:21.453557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:21.454551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.455159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.455435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.455705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.456359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:21.456854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:21.458341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:21.461089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 107/214] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:21.462454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.469023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 108/215] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:21.477527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.575302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea1e06-8c0f-4cdf-87b3-77d9a990ac56 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.584723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:21.619880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f833f-6c66-4a98-9ea7-ae43456b3006 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.620181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.620990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.621368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.621740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.622671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:21.625088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:21.654912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 108/216] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:21 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:38:21.666325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.667285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.667726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.668211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] PATH_INFO: `/v3/users/945010dbc0ad4d42864716b0a35736df` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.687367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] RBAC: Authorizing `identity:get_user(user_id=945010dbc0ad4d42864716b0a35736df)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:21.689303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:21.691726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 109/217] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:21 2022] GET /identity/v3/users/945010dbc0ad4d42864716b0a35736df => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:21.697769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:21.766686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:21.819198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c623f-988a-4f35-b439-61981b34f916 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:21.858108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:21.858698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:21.858935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:21.859163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:21.932322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:21.933820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:21.992241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:22.041792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 109/218] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:22.049075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:22.181842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47d2e-386a-4995-b28e-61796c691d89 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:22.219722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:22.226536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:22.227159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.227805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.228087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.229365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:22.230905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d28cc-800a-4d17-8e00-ec6822fc0ed8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:22.238142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 110/219] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:22 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:22.239724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:22.393764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57814f6c-63a0-4a3b-ba9a-89b7b5c38454 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:22.446390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:22.447352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.447833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.448296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.449244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=945010dbc0ad4d42864716b0a35736df) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:22.449762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:22.451385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b874f85-afd6-4066-a71f-98380a75cbe6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:22.461175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 110/220] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:22 2022] GET /identity/v3/role_assignments?user.id=945010dbc0ad4d42864716b0a35736df&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=3fbe9323075347109c2f57834701d0de => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:22.602103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2096-ace8-493c-9afa-7235c4e58c3e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.602468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2096-ace8-493c-9afa-7235c4e58c3e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.602667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2096-ace8-493c-9afa-7235c4e58c3e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.604289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 111/221] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:22.609937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.610213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.610492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.629993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:22.631213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:22.647426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:22.670221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe2ab0-66cb-42d7-ab90-ae0c5bd99964 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.670995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe2ab0-66cb-42d7-ab90-ae0c5bd99964 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.671325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe2ab0-66cb-42d7-ab90-ae0c5bd99964 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.673031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 112/222] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:22.682482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:22.682800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:22.683023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:22.715082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:22.717770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:22.737892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 111/223] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:22.815207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 113/224] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:23.041443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.158070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0b7cd-0fc8-4d9a-8b50-5703381ef509 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.161389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.196012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.197075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.198107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.210842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:23.216240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:23.217231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:23.218033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:23.220868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 112/225] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:23.227991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.242907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:23.340070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba07f-181f-4e2c-8802-69cb47e44397 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.372384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:23.379352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.380237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.380843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.381423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.392806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:23.398686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:23.399853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:23.400708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:23.403074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 114/226] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:23.408857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.434785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0e81a-b73d-42bb-a420-181371f86557 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.514428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.515185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.515391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.515955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.516643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:23.516953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:23.518413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:23.522237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623f488-8f6a-412e-b80f-aa74aaedd945 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.536125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 113/227] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:23.557597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.566836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.567745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.568243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.568743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.569639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:23.570134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:23.571812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:23.582609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 115/228] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:23.589589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.672920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:23.702123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:23.753330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f678dc-f9a6-4bf2-9eaf-21be5a7ad691 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.765897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973ca6e-f821-4aa6-ba83-57238c0bf955 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:23.817353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.818554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.819157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.819486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.821306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:23.822743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:23.824306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 114/229] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:23.828083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:23.828745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:23.829024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:23.829335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:23.830701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:23.831683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:23.833156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:23.848305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 116/230] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:23.849213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:24.006520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705d8f-b5db-4b6a-85ec-bc235aaf0146 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:24.015579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6f527-ee94-48b5-88da-e060d0dc914b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:24.028830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:24.044981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:24.045980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.046459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.046939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.048240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:24.049960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:24.051973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 117/231] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:24.059256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:24.087269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:24.087806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.088029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.088273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.089439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:24.090909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:24.096632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 115/232] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:23 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:24.106244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:24.134988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:24.181657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815ae87-1365-4e40-b1ee-cbb0c39e628f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:24.218674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:24.219516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.219893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.220264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/f714b6b7421c4d6db5f5f173dc172c0f/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.223051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, user_id=f714b6b7421c4d6db5f5f173dc172c0f, role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:24.225962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9124658-5651-49c1-ad4f-153f08b3120c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:24.244565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 118/233] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:24 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/f714b6b7421c4d6db5f5f173dc172c0f/roles/d53450bb086544ce968b603c0997c5b9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:24.327533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f169d04-36c0-48e0-b561-749f74d819eb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:24.379636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:24.380240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfdc2-3cc3-46f2-92c8-bdbb6319292d None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.380351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.380580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.380881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/f714b6b7421c4d6db5f5f173dc172c0f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.381287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfdc2-3cc3-46f2-92c8-bdbb6319292d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.381755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bfdc2-3cc3-46f2-92c8-bdbb6319292d None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.383405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, user_id=f714b6b7421c4d6db5f5f173dc172c0f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:24.384392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 119/234] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:24.388691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:24.390376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc75ad-c484-44fe-aef8-3ef5f411c183 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.390872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc75ad-c484-44fe-aef8-3ef5f411c183 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.391342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc75ad-c484-44fe-aef8-3ef5f411c183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.416118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0fcc75ad-c484-44fe-aef8-3ef5f411c183 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:24.417861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0fcc75ad-c484-44fe-aef8-3ef5f411c183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:24.421245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 116/235] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:24 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/f714b6b7421c4d6db5f5f173dc172c0f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:38:24.476302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:24.519622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 120/236] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:24.821738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:24.913352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:24.955525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1abdf-fcd6-4a9b-9e31-c17c9efde252 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:24.969097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293753-4ded-4913-81aa-38f53978c806 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.969424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293753-4ded-4913-81aa-38f53978c806 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.969889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01293753-4ded-4913-81aa-38f53978c806 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.972097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 121/237] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:24.978018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dd1de-2fd8-4d11-acf6-1172475918d5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:24.978365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dd1de-2fd8-4d11-acf6-1172475918d5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:24.978711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dd1de-2fd8-4d11-acf6-1172475918d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:24.998068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-984dd1de-2fd8-4d11-acf6-1172475918d5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:24.999411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-984dd1de-2fd8-4d11-acf6-1172475918d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:25.009261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.009904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.010181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.010446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.032766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:25.039737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:25.040672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:25.041503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:25.046577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 117/238] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:25.052267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.120979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 122/239] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:25.203571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7064-13fc-4f7c-92e8-5c274f67503b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.204169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7064-13fc-4f7c-92e8-5c274f67503b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.204169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a7064-13fc-4f7c-92e8-5c274f67503b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.205525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 123/240] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:25.213537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.213820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.214100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.239885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:25.241105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:25.261960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0cbc5-39e6-4c45-8c6d-c33ad7aadd2b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:25.308915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:25.325817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.326431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.326709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.326978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.327651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:25.327955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:25.329368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:25.344697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 118/241] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:25.351081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.385465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 124/242] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:25.461232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.531740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:25.544944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc17bb-39b7-426d-8c36-5b60e02b6002 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:25.611495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.612155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.612438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.612751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] PATH_INFO: `/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.638261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7966a7ba43a443ae92144f7ce7f9ce3b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:25.639670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:25.641378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 119/243] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:25 2022] GET /identity/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:25.646282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cc01-86b0-4259-9785-52a75de7551b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:25.648977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.711440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.713538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.713821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.714095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.721328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:25.722791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:25.724716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 125/244] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:25.728995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:25.730615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.792982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8ee5b-5ed2-4161-a58d-644ee3a6a7d1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:25.839166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.839928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.840460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.840961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.853974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfae9c4-a3e3-4287-b90e-6b55f672758a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:25.863733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:25.895188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:25.896279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:25.896761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:25.897172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:25.898571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:25.900427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:25.923827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:25.925402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 126/245] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:25 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:25.925871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:25.926566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:25.993355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:26.063396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84dc4a-3cdb-48a3-b2b1-04de6cd56906 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.106670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.107474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.107861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.108234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.108928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 120/246] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:26.114688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.122016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:26.134145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:26.135197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:26.136033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 127/247] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:26.141188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.179088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:26.284799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5906-aa91-477c-933d-c0bb3cdc7054 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.306343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cf78d-1d5d-4559-9502-8d1e2b916106 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.331398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.332654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.333572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.334054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.335723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:26.336493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.338848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.345066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.345694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.345972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.346239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.347359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.348842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.350788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 128/249] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:26.351382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 121/249] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:26 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:26.356937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.357259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.380790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:26.470270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:26.489713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e034ddd-c351-41c0-acd9-10ece125f380 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.489900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2739c-251f-4197-a3d8-ba571b2a3a66 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.529778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.530389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.530663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.530978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.531697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=7966a7ba43a443ae92144f7ce7f9ce3b) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:26.532574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.534744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.538920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.539565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.539853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.540128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] PATH_INFO: `/v3/users/945010dbc0ad4d42864716b0a35736df` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.542816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] RBAC: Authorizing `identity:get_user(user_id=945010dbc0ad4d42864716b0a35736df)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.544304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.546586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 122/250] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:26 2022] GET /identity/v3/users/945010dbc0ad4d42864716b0a35736df => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:26.547846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 129/251] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:26 2022] GET /identity/v3/role_assignments?user.id=7966a7ba43a443ae92144f7ce7f9ce3b&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=7bf924eb503449ccaddc82aad1e450af => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:26.559234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.671424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:26.689411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628ebf1-a627-489c-aec7-21b301607673 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.736211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.736848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.737131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.737476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.738654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.740151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a55d59-b2a0-439e-a350-c74b918f61e7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.741992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 123/252] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:26 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:26.748130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:26.858706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f304f-4723-4ec7-9733-fd41b1e6193c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:26.910025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:26.910712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:26.910991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:26.911265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/945010dbc0ad4d42864716b0a35736df/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:26.913907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, user_id=945010dbc0ad4d42864716b0a35736df, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:26.915387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011ee36-6b75-415a-8405-f9945c00b3bc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:26.932523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 130/253] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:26 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/945010dbc0ad4d42864716b0a35736df/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:27.019541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d67c-fb93-4478-b69b-ad5236b3fa8c None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.019884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d67c-fb93-4478-b69b-ad5236b3fa8c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.020097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d67c-fb93-4478-b69b-ad5236b3fa8c None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.021859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 124/254] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:27.027452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780f80-fbbb-40c3-9f00-942a9399cae0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.027739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780f80-fbbb-40c3-9f00-942a9399cae0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.028010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55780f80-fbbb-40c3-9f00-942a9399cae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.048118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-55780f80-fbbb-40c3-9f00-942a9399cae0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:27.049373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-55780f80-fbbb-40c3-9f00-942a9399cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:27.051568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:27.136580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 131/255] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:27.175434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266309-1f5f-470f-826b-8384021c7cc5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.175728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266309-1f5f-470f-826b-8384021c7cc5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.175995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266309-1f5f-470f-826b-8384021c7cc5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.177390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 125/256] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:27.183704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.183984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.184258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.207349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:27.208623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:27.298888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 132/257] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:27.471257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c266309-1f5f-470f-826b-8384021c7cc5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:27.609938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c266309-1f5f-470f-826b-8384021c7cc5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:27.661315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:27.661959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.662239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.662509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.676274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:27.681192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:27.682115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:27.682933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 126/258] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:27.689668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:27.753656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:27.777322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:27.777921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:27.829129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29c7ec-063d-49f5-b504-e5a86fdfdc14 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:27.880347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:27.880912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:27.881190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:27.881460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:27.882165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:27.882468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:27.883853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:27.892501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 133/259] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:27.909410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:27.951273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a95263-300c-4791-bfbc-66be272a7265 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.020134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.021269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.021791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.022252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.041531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:28.046891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:28.047769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:28.048673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:28.051259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 127/260] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:28.057991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.088570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5b222-8a9f-42f2-8d78-1d65c5dc774d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.166202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:28.181829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.182495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.182775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.183044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.186173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:28.187678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:28.192730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 134/261] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:27 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:28.203657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.223733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:28.276632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c329d80-14ef-40ee-992d-82f89b0a9dca None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.334361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.335530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.335810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.336079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.336872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:28.337173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:28.338698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:28.356626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 128/262] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:28.362431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.406269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c19006-1a12-429f-b962-7593db61a004 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.457935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.458471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.458751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.459028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.540841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247c3d4-089c-47e4-8b7d-7045eaf96dd5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.553860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:28.559933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:28.561470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:28.645896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.646600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.646881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.647155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] PATH_INFO: `/v3/users/f714b6b7421c4d6db5f5f173dc172c0f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.648373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] RBAC: Authorizing `identity:get_user(user_id=f714b6b7421c4d6db5f5f173dc172c0f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:28.650191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:28.652113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 129/263] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/users/f714b6b7421c4d6db5f5f173dc172c0f => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:28.657509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.666348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:28.723310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 135/264] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:28.729052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.860397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:28.868124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24edd0-3fd4-4bee-9f13-ebb717b2a5d6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.912689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6909ab-4d1b-4287-a002-30b78e6180cf None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:28.935243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.935988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.936268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.936539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:28.939183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:28.940649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:28.942715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 136/265] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:28.948814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:28.980231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:28.980704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:28.980995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:28.981268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.080813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:29.083010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:29.095038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:29.178802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8df22-b5a5-42dd-aacb-34e3d6478a23 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:29.253890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:29.254525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:29.254809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:29.255087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.255822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=f714b6b7421c4d6db5f5f173dc172c0f) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:29.256133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:29.257632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:29.266690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 137/266] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/role_assignments?user.id=f714b6b7421c4d6db5f5f173dc172c0f&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=d53450bb086544ce968b603c0997c5b9 => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:29.301724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 130/267] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:29.307414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:29.349693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:29.448051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f42385-848c-4ab0-8d15-19be6e88a405 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:29.486083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:29.486749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:29.487027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:29.487304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.488482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:29.489990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cc3ce-4275-460b-a330-9e18d5ffd699 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:29.492626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 138/268] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:29 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:29.498717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:29.614984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61779da-a78f-4626-ab63-e4ddfe20820d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:29.616734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:29.644996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab319-c176-436a-b491-ac55c2a08255 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:29.645342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab319-c176-436a-b491-ac55c2a08255 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:29.645546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ab319-c176-436a-b491-ac55c2a08255 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.646883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 139/269] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:29.654825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acfd56-c744-4acc-9110-1bac2d73ee12 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:29.655098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acfd56-c744-4acc-9110-1bac2d73ee12 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:29.655370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acfd56-c744-4acc-9110-1bac2d73ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.681749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-81acfd56-c744-4acc-9110-1bac2d73ee12 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:29.682965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-81acfd56-c744-4acc-9110-1bac2d73ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:29.687631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:29.688250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:29.688530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:29.688836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:29.689564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=f714b6b7421c4d6db5f5f173dc172c0f) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:29.689868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:29.691296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bd744-ff23-47a8-8d40-5436e70ede12 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:29.706940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 131/270] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:29 2022] GET /identity/v3/role_assignments?user.id=f714b6b7421c4d6db5f5f173dc172c0f&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:29.793923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 140/271] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:30.027762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e23-fb06-4ef9-a8ec-80d39ea1df1e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.028038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e23-fb06-4ef9-a8ec-80d39ea1df1e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.028578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa81e23-fb06-4ef9-a8ec-80d39ea1df1e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.030257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 132/272] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:30.036426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815d58-8e35-4c2e-98fc-f84bacf820c4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.037110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815d58-8e35-4c2e-98fc-f84bacf820c4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.037596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d815d58-8e35-4c2e-98fc-f84bacf820c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.061241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-2d815d58-8e35-4c2e-98fc-f84bacf820c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:30.062760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2d815d58-8e35-4c2e-98fc-f84bacf820c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:30.094023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.154617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 141/273] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:30.180363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf6ef6-4495-4cf8-84e2-83439f324f2e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.180945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf6ef6-4495-4cf8-84e2-83439f324f2e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.181427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebf6ef6-4495-4cf8-84e2-83439f324f2e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.183194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 133/274] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:30.188797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b431f-5d31-4e66-a5db-74ff41131659 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.189279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b431f-5d31-4e66-a5db-74ff41131659 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.189728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b431f-5d31-4e66-a5db-74ff41131659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.204542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf6ef6-4495-4cf8-84e2-83439f324f2e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.211223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b74b431f-5d31-4e66-a5db-74ff41131659 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:30.212428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b74b431f-5d31-4e66-a5db-74ff41131659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:30.296713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 142/275] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:30.296713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9fd1f-0478-48fa-8a6c-f42565dc85b6 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.296713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9fd1f-0478-48fa-8a6c-f42565dc85b6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.296713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9fd1f-0478-48fa-8a6c-f42565dc85b6 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.296713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 143/276] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:30.298475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.298757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.299032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.317974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:30.319823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:30.324093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebf6ef6-4495-4cf8-84e2-83439f324f2e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.364216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.365024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.365404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.365776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.379673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:30.387556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:30.388509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:30.389338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:30.390044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 134/277] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:38:30.395130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.397204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 144/278] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:30.424530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.433756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.453963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.519592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba7217-a06c-405b-a59a-8b653245c27f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.549086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4bed0-758d-4b60-9460-9f13c475dcdd None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.562967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.563577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.564103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.564377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.565521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:30.565945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:30.567705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:30.579168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 135/279] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:30.584285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.589715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.590394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.590671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.590934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.592016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:30.593473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:30.595396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 145/280] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:30.602550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.681301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.690584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.695357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bcd1d-c715-4851-929c-c2b44196cb3f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.707339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29be541-dfd6-470d-99a5-b1fc8540f338 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.729360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.729984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.730273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.730545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] PATH_INFO: `/v3/users/925b7553049849688e8243b1a1d07c38` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.741590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.742187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.742463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.742727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.743366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:30.743454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=925b7553049849688e8243b1a1d07c38)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:30.744816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:30.744909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:30.746466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 136/281] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/users/925b7553049849688e8243b1a1d07c38 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:30.748121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.765713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 146/282] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:30 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:30.766540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.858726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cafd5-311b-4cee-8f8c-a84941203bc8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.890257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0fab4-cb0a-407a-9366-ac4ab69d3abe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:30.900898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.901548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.902252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.902528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.914355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:30.919322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:30.920294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 137/283] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:30.925458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.935919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:30.937197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:30.938214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:30.938489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:30.938760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:30.952031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:30.957678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:30.958550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 147/284] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:30.959358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:30.961955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.049378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2a243-90b9-4fb9-b741-628b1891fe5f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.088891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29dc8b-c3dd-4305-a71b-40af1741752d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.093521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.094138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.094417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.094683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.095416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, user.id=925b7553049849688e8243b1a1d07c38, scope.system=all) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:31.095724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.097665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.105208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 138/285] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/role_assignments?user.id=925b7553049849688e8243b1a1d07c38&scope.system=all&role.id=3fbe9323075347109c2f57834701d0de => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:31.106264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.128038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.128676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.128962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.129230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.129918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:31.130220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.131665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.145706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 148/286] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:31.151756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.223637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.262767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687045eb-95f3-4f58-8fb8-d66acfcb0ba9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.305330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.313310 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.314296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.314800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.315268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.318057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:31.318750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.321626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.342588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 139/287] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:31.349790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.362536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cd70d-30ba-40bf-97fa-a0ba0daf7a95 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.404997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.405555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.405837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.406110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] PATH_INFO: `/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.410248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] RBAC: Authorizing `identity:get_user(user_id=7966a7ba43a443ae92144f7ce7f9ce3b)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.411708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.413953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 149/288] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:31 2022] GET /identity/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:31.425606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.488813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36bddb-306d-4adb-b845-155195a6299a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.519171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.530897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.531537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.531814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.532084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] PATH_INFO: `/v3/users/945010dbc0ad4d42864716b0a35736df` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.533845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] RBAC: Authorizing `identity:get_user(user_id=945010dbc0ad4d42864716b0a35736df)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.535304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.537297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 140/289] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:31 2022] GET /identity/v3/users/945010dbc0ad4d42864716b0a35736df => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:31.543003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.615231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb071b-345f-44b9-8b8f-7ca5a581e445 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.668302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.675230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.675871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.676160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.676436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.678458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.682102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.684217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 150/290] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:31 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:31.690772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:31.706374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f88d-c6c9-4ce1-b497-2bbaa9dfa060 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.772042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.773222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.773502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.773773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.868415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:31.873766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.875248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.896397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67612280-eaaa-4b02-a102-cdbbfbfc4f64 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:31.943273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:31.943856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:31.944133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:31.944436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/7966a7ba43a443ae92144f7ce7f9ce3b/roles/7bf924eb503449ccaddc82aad1e450af` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:31.946393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, user_id=7966a7ba43a443ae92144f7ce7f9ce3b, role_id=7bf924eb503449ccaddc82aad1e450af)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:31.948288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:31.987331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 151/291] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:38:31 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/users/7966a7ba43a443ae92144f7ce7f9ce3b/roles/7bf924eb503449ccaddc82aad1e450af => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:38:32.057457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 141/292] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 20 04:38:32.062893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:32.159440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:32.217258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13bcde-5368-4104-98d5-24b585ccb4b2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:32.265820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:32.266465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:32.266745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:32.267017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:32.268141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:32.269605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb5af3-e977-43ff-a1a7-c2fc7b8ebb0a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:32.271240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 152/293] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:32 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:32.284549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:32.410426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:32.417078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed300d3-edd5-47aa-85e5-8bf8024a349f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:32.470738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:32.471268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:32.471548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:32.471815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:32.472547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=945010dbc0ad4d42864716b0a35736df) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:32.472883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:32.474312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c78cf-46b7-479c-a478-0cc8aa8df6a4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:32.490968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 142/294] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:32 2022] GET /identity/v3/role_assignments?user.id=945010dbc0ad4d42864716b0a35736df&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=3fbe9323075347109c2f57834701d0de => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:33.390291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb313fb-f974-4fa2-ada9-337bf8aafbb1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.391352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb313fb-f974-4fa2-ada9-337bf8aafbb1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.391844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb313fb-f974-4fa2-ada9-337bf8aafbb1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.393536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 153/295] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:33.399889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb889-0294-476e-9ef2-ecd371129e6d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.400877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb889-0294-476e-9ef2-ecd371129e6d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.401378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bb889-0294-476e-9ef2-ecd371129e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.420747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d20bb889-0294-476e-9ef2-ecd371129e6d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:33.422313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d20bb889-0294-476e-9ef2-ecd371129e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:33.436044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2e64f-4f5b-48f1-a3ba-d720bb2eed12 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.436320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2e64f-4f5b-48f1-a3ba-d720bb2eed12 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.436582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2e64f-4f5b-48f1-a3ba-d720bb2eed12 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.438390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 154/296] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:33.450073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039426-ce98-4605-a811-af79946e950c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.450737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039426-ce98-4605-a811-af79946e950c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.451213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039426-ce98-4605-a811-af79946e950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.471680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-97039426-ce98-4605-a811-af79946e950c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:33.473222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-97039426-ce98-4605-a811-af79946e950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:33.504627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 143/297] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:33.505941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4587a-8647-4bc6-81e4-a05e6e32517d None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.506232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4587a-8647-4bc6-81e4-a05e6e32517d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.506506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4587a-8647-4bc6-81e4-a05e6e32517d None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.508743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 144/298] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:33.514453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.514931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.515353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.534964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:33.536386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:33.555902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 155/299] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:33.606847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 145/300] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:33.777180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039426-ce98-4605-a811-af79946e950c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:33.830726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:33.843834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:33.863634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:33.896377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039426-ce98-4605-a811-af79946e950c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:33.931901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:33.932630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.932924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.933627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.945278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:33.945528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ea35c-734e-497f-b27f-26b29b8a525f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:33.950150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:33.951033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:33.951860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 156/301] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:33.952521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:33.981711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:33.982583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:33.982858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:33.983128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:33.984214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:33.985684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:33.987560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 146/302] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:33.988433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.063936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11ea6-a43d-40b4-bdc0-c382f6d5751e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.084200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.086231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.094705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc997-c749-4069-9db3-113cd9252299 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.104866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.105499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.105928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.106035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.116935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:34.121724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:34.122600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:34.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 157/303] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:34.124081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.144139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.145268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.145713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.146356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.148163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:34.148619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.150376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.162576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 147/304] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:34.165648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.239701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76357-c683-4241-a1c3-2acb328172d1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.272631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.273487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.273773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.274041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] PATH_INFO: `/v3/groups` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.274912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.276340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.280135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e80b6-58e4-4987-8ac5-b384e4ced0e8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.294468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 158/305] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Sep 20 04:38:34 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:34.295494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.332081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.332794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.333268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.333798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.334739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:34.335705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.337447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.352619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.353201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 148/306] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:34.355050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.376467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.460658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ee5d5-eabf-46e0-8b4b-a1a09d0b0445 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.473462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad92b7-f22d-41ab-b6cc-82c88a886222 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.511966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.512166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.512722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.513000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.513269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] PATH_INFO: `/v3/users/79d02b4e3d074594aab27bc1319938e2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.513485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.513900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.514307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] PATH_INFO: `/v3/users/925b7553049849688e8243b1a1d07c38` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.515762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] RBAC: Authorizing `identity:get_user(user_id=925b7553049849688e8243b1a1d07c38)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.518086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.524969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 149/307] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:34 2022] GET /identity/v3/users/925b7553049849688e8243b1a1d07c38 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:34.526817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.535190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=79d02b4e3d074594aab27bc1319938e2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.536722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.538961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 159/308] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:34 2022] GET /identity/v3/users/79d02b4e3d074594aab27bc1319938e2 => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:34.545251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:34.627730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.644044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:34.650990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270180-5584-480b-baf9-cc57f2147e8e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.651261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb89b9-f7dd-4402-add0-cf832f4eb2e0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:34.686695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.686921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:34.687349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.687451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.687627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.687730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.687900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] PATH_INFO: `/v3/system/users/925b7553049849688e8243b1a1d07c38/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.687992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.688684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, user.id=79d02b4e3d074594aab27bc1319938e2, scope.system=all) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:34.688985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.689469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=925b7553049849688e8243b1a1d07c38, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:34.690399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86982e8b-dd71-405f-9d10-279138f0641f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.690835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:34.696086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 160/309] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Sep 20 04:38:34 2022] GET /identity/v3/role_assignments?user.id=79d02b4e3d074594aab27bc1319938e2&scope.system=all&role.id=7bf924eb503449ccaddc82aad1e450af => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:34.697489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5e1a-fc6a-423c-84aa-5dde77e114d9 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.697738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5e1a-fc6a-423c-84aa-5dde77e114d9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.697999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5e1a-fc6a-423c-84aa-5dde77e114d9 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.699350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 161/310] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:34.705053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:34.705294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:34.705597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:34.708038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 150/311] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Sep 20 04:38:34 2022] PUT /identity/v3/system/users/925b7553049849688e8243b1a1d07c38/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:38:34.734746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:34.736039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:34.829803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 162/312] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:35.118112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:35.233792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df779e44-44a5-40e3-a4ed-761f6d007d96 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:35.269413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:35.270388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:35.270912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:35.271389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:35.283039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:35.289286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:35.290239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:35.291043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:35.293862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 151/313] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:35.300075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:35.405912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:35.415398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c46c4-e626-4139-b66c-f6083b22f8cc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:35.451494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:35.452901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:35.453405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:35.453876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:35.454736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:35.455236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:35.456828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:35.466660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 163/314] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:35.473071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:35.484906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:35.582592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e91dc-3095-4ed1-b44f-c2b05540c4ac None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:35.622328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:35.623368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:35.623852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:35.624323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] PATH_INFO: `/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:35.625707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] RBAC: Authorizing `identity:get_user(user_id=7966a7ba43a443ae92144f7ce7f9ce3b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:35.627381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:35.629612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 152/315] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:35 2022] GET /identity/v3/users/7966a7ba43a443ae92144f7ce7f9ce3b => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:35.636169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:35.750136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18b974-433e-42df-865b-c28b23ae18ff None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:35.792920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:35.793520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:35.793796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:35.794063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:35.815750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:35.863227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:35.864701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:35.963482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 164/316] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:35.969124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:36.073087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:36.081302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c69262-3e69-4e21-9097-477fec520a78 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:36.117270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:36.118245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.118747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.119221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.120536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:36.122229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c304248-7d33-455b-8f27-85f9fdd0663f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:36.124345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 153/317] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:35 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:36.133708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:36.187563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:36.236296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3c6a-0d9d-4e60-b4b8-cbe20d33d7c5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:36.270695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:36.271303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.271582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.271850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.273195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6, user.id=7966a7ba43a443ae92144f7ce7f9ce3b) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:36.273547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:36.275253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0c294-33c0-462f-9259-0b7dcea614ca None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:36.281462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 165/318] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:38:36 2022] GET /identity/v3/role_assignments?user.id=7966a7ba43a443ae92144f7ce7f9ce3b&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=7bf924eb503449ccaddc82aad1e450af => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:36.637761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18acae-3fed-4c79-b3a0-35b703de9746 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.638276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18acae-3fed-4c79-b3a0-35b703de9746 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.638378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18acae-3fed-4c79-b3a0-35b703de9746 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.639836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 154/319] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:36.644786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cba61-3ebb-4901-bbdb-5428a31256c8 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.645161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cba61-3ebb-4901-bbdb-5428a31256c8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.645326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cba61-3ebb-4901-bbdb-5428a31256c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.664829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-090cba61-3ebb-4901-bbdb-5428a31256c8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:36.666010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-090cba61-3ebb-4901-bbdb-5428a31256c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:36.730501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 166/320] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:36.928185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b97dc-e01e-484d-8d09-b3f5de770b49 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.929010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b97dc-e01e-484d-8d09-b3f5de770b49 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.929244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b97dc-e01e-484d-8d09-b3f5de770b49 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.930582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 155/321] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:36.935624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.935917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.936190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.954480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:36.955661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:36.985952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7086c-65e7-43a4-ac31-8a4b0dbae6f9 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.986464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7086c-65e7-43a4-ac31-8a4b0dbae6f9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.986572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7086c-65e7-43a4-ac31-8a4b0dbae6f9 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.987846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 156/322] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:36.993318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:36.993597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:36.993865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:36.999902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.011406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:37.012578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:37.023897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 167/323] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:37.025611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.089012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 157/324] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:37.134453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183bc9d-d820-41b4-9ed8-b7ca37979e46 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.173013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.173635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.173908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.174171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.175299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.176760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.180952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 168/325] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:37.184324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.259847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.285880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d2362-12b7-4e3d-bdf9-d003ccf0d4bc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.290738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.324081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.324702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.324977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.325243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] PATH_INFO: `/v3/groups` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.325873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.327328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.338857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.347530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 158/326] 10.222.0.27 () {60 vars in 1134 bytes} [Tue Sep 20 04:38:37 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:37.364034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.404391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f99647-1e7e-4714-bd8a-2fcbfdb46da5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.448477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.449580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.450052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.450517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.462985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:37.468249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:37.469326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:37.470307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:37.472623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 169/327] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:38:37.478864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.489370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7d43f-90a8-4fc7-8b5e-cb72c79df0d2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.531917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.532556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.532869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.533866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.535010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.549807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:37.554844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:37.555934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:37.557029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:37.560962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 159/328] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:38:37.561833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.598664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c950e-6f64-4a4a-8b24-de629e0555dd None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.604515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.641397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.642015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.642295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.642566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.643236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:37.643535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.644942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.655807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 170/329] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:37.661773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.682774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91570a4-f42a-4aaa-b44b-755c88ecc58d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.739396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.740246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.740550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.740837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.742062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:37.742366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.743753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.753755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 160/330] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:37.759522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.788432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0adbd-c064-4bf2-8094-2120e3fa721e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.828427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.837815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.838789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.839263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.839725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] PATH_INFO: `/v3/users/925b7553049849688e8243b1a1d07c38` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.841138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] RBAC: Authorizing `identity:get_user(user_id=925b7553049849688e8243b1a1d07c38)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.842815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.846031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 171/331] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/users/925b7553049849688e8243b1a1d07c38 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:37.855696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.886554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b53a-b26a-4e09-b975-ba05c16a104a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:37.905055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:37.936478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:37.937669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:37.938135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:37.938480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] PATH_INFO: `/v3/users/79d02b4e3d074594aab27bc1319938e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:37.939705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] RBAC: Authorizing `identity:get_user(user_id=79d02b4e3d074594aab27bc1319938e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:37.941955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:37.944222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 161/332] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/users/79d02b4e3d074594aab27bc1319938e2 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:37.950793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:37.982634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971859-b68a-4136-86d5-039b29b79445 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:38.027015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:38.027958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:38.028426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:38.028915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:38.029826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] RBAC: Adding query filter params (role.id=3fbe9323075347109c2f57834701d0de, user.id=925b7553049849688e8243b1a1d07c38, scope.system=all) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:38.030289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:38.033839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420afba0-f526-4db5-b6a4-75d3f9485321 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:38.054501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 172/333] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Sep 20 04:38:37 2022] GET /identity/v3/role_assignments?user.id=925b7553049849688e8243b1a1d07c38&scope.system=all&role.id=3fbe9323075347109c2f57834701d0de => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:38.093709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6945727-fadf-46f6-9035-bf3b95861be1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:38.121618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:38.135034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:38.137421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:38.137709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:38.137984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] PATH_INFO: `/v3/system/users/79d02b4e3d074594aab27bc1319938e2/roles/7bf924eb503449ccaddc82aad1e450af` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:38.139681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=79d02b4e3d074594aab27bc1319938e2, role_id=7bf924eb503449ccaddc82aad1e450af)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:38.141158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8591cc6-c7ee-4844-a1f0-b48ae038acbb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:38.155188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 162/334] 10.222.0.27 () {58 vars in 1334 bytes} [Tue Sep 20 04:38:37 2022] PUT /identity/v3/system/users/79d02b4e3d074594aab27bc1319938e2/roles/7bf924eb503449ccaddc82aad1e450af => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:39.871863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ac1-aa4d-49cc-8336-2fcd4b0cee8a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:39.872381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ac1-aa4d-49cc-8336-2fcd4b0cee8a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:39.872381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61ac1-aa4d-49cc-8336-2fcd4b0cee8a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:39.873902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 173/335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:39.879107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:39.879571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:39.879723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:39.902942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:39.904138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:39.989602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 163/336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:40.317424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61ac1-aa4d-49cc-8336-2fcd4b0cee8a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:40.424449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:40.434518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61ac1-aa4d-49cc-8336-2fcd4b0cee8a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:40.477015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:40.477649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:40.477926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:40.478196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:40.488505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:40.494536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:40.497097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:40.499088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 174/337] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:40.504623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:40.642252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fca97-749f-4876-aaa2-e39e8feb12ff None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:40.668729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:40.685100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:40.685644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:40.686027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:40.686258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:40.686905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:40.687206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:40.688592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaaf19-45c3-4cbf-b21f-147b2430d5ad None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:40.701256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 164/338] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:40.707988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:40.828194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317a8a6-f088-4231-8199-28bcfa65adab None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:40.870037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:40.870680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:40.870960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:40.871237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:40.894734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:40.952155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:40.953600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ced982-e9bf-489b-aaa0-826dab597b8b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:40.974093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffee27-35f8-4fdd-b36d-df83b216e9d4 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:40.974707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffee27-35f8-4fdd-b36d-df83b216e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:40.974707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffee27-35f8-4fdd-b36d-df83b216e9d4 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:40.976835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 165/339] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:40.981873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25148f-a136-47ca-87a2-17faa4892a68 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:40.982161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25148f-a136-47ca-87a2-17faa4892a68 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:40.982442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25148f-a136-47ca-87a2-17faa4892a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.017619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-8e25148f-a136-47ca-87a2-17faa4892a68 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:41.018852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8e25148f-a136-47ca-87a2-17faa4892a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:41.129043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 175/340] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:41.130558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d187-e4f7-4f7d-bfea-8c3fe0ce72dd None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.130842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d187-e4f7-4f7d-bfea-8c3fe0ce72dd None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.131115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d187-e4f7-4f7d-bfea-8c3fe0ce72dd None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.132553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 176/341] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:41.137664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878d187-e4f7-4f7d-bfea-8c3fe0ce72dd None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:41.161770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 166/342] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:41.163533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a5c6b-56b3-4241-9e02-99267f000e6f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.163884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a5c6b-56b3-4241-9e02-99267f000e6f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.164163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a5c6b-56b3-4241-9e02-99267f000e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.195555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ea1a5c6b-56b3-4241-9e02-99267f000e6f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:41.196840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ea1a5c6b-56b3-4241-9e02-99267f000e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:41.253564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:41.285501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 167/343] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:41.327662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878d187-e4f7-4f7d-bfea-8c3fe0ce72dd None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:41.343224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1fd04-1551-4bee-ab85-8fc5899401f9 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.343504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1fd04-1551-4bee-ab85-8fc5899401f9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.343770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1fd04-1551-4bee-ab85-8fc5899401f9 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.345398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 168/344] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:41.361959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.362240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.362518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.370003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:41.370631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.370949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.371228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.372342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:41.374761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:41.382726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 177/345] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:41.386283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:41.402330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:41.403515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:41.458186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:41.495742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 169/346] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:41.508308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29462f3-6125-4005-a62c-43982b6b3f4d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:41.531224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:41.548405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:41.549078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.549353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.549623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.555917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:41.557555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:41.559187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 178/347] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:41.565164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:41.726630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26cd38-e1b9-43e5-91c3-435a540dc293 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:41.738302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:41.773416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:41.773988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.774388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.774927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.775665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=e9483c5eaf14474197c855467cfe5a7d) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:41.775971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:41.777441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:41.783368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 170/348] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=3fbe9323075347109c2f57834701d0de => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:41.887530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:41.916636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593aa56-5bee-4d6e-b04c-69e88747731f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:41.953335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:41.964341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:41.965068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:41.965290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:41.965564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:41.983653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:41.991658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:41.992677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:41.993492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 179/349] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:41.994168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.059563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.091469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0573af-ac6a-4906-95a2-6a68863738aa None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.130359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572a03c-57ee-4e1b-8b7c-1aec6586cd99 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.146646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.147852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.148136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.148409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.173025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:42.181016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:42.181951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:42.182789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 171/350] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:38:42.183482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.185082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.185961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.186555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.187048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.201204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:42.207594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:42.208528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 20 04:38:42.210310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:42.212515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 180/351] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:42.213591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.319826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.331184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434433b0-d825-44ca-ba67-1187844c39f0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.342918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6a984-8268-484e-8e23-74e36206f155 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.346876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.391182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.391902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.392260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.392639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.393399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:42.393776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:42.395554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:42.406477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 172/352] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:42.407386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.415380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.417261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.417842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.418298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.420084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:42.420653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:42.422278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:42.433970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 181/353] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:42.435123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.541677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4b785-6ea4-4f35-a798-2a725880e473 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.559815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b3e3c-f628-4fe5-981a-3c5f41309673 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.590478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.591135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.591455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.591803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.592561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:42.593013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:42.594688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:42.599226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a64e0-b070-473f-928b-3226a084168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.600095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a64e0-b070-473f-928b-3226a084168a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.601223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a64e0-b070-473f-928b-3226a084168a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.601757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a64e0-b070-473f-928b-3226a084168a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.604884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 173/354] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:42.605671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.632474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.650436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.704405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a64e0-b070-473f-928b-3226a084168a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:42.706813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a64e0-b070-473f-928b-3226a084168a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:42.738031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f651125-ac88-4010-a0df-9d76d6cb5298 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.781688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:42.782305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.782588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.782863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.847618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 182/355] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:42.848843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6327-ca9d-451c-b303-a4f22f5c4755 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.849124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6327-ca9d-451c-b303-a4f22f5c4755 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.849390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f6327-ca9d-451c-b303-a4f22f5c4755 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.852009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 183/356] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:42.853165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca357-5157-46d4-ba8c-9203b7fa98a8 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:42.853443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca357-5157-46d4-ba8c-9203b7fa98a8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:42.853714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcca357-5157-46d4-ba8c-9203b7fa98a8 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:42.854906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 184/357] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:42.855663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca357-5157-46d4-ba8c-9203b7fa98a8 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:42.873949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:42.875565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:42.894197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.939315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:42.983971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcca357-5157-46d4-ba8c-9203b7fa98a8 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:42.993572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 174/358] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:42.994334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.030543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.031250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.031474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.032103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] PATH_INFO: `/v3/users/79d02b4e3d074594aab27bc1319938e2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.033529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] RBAC: Authorizing `identity:get_user(user_id=79d02b4e3d074594aab27bc1319938e2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.035621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af8e49-0d9b-4d81-a264-37fc03a13cbc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.039092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 185/359] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/users/79d02b4e3d074594aab27bc1319938e2 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.040718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fb126-905d-4927-8513-b458b06fc586 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.041102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fb126-905d-4927-8513-b458b06fc586 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.041202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fb126-905d-4927-8513-b458b06fc586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.065077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9e3fb126-905d-4927-8513-b458b06fc586 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:43.066563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9e3fb126-905d-4927-8513-b458b06fc586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:43.134844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c6785-2a36-4c57-8248-535d7ad55542 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.158541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.169092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 186/360] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:43.170568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.170686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.170912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.178863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.179784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.180213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.180662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.181946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.183572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.185781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 175/361] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:42 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.186844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.189144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:43.190451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:43.287195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.297669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 187/362] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:43.298090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.351068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de72f6-c4ac-4203-8e5c-2596e8a4cafb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.421288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.421836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.422112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.422382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.425182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.426768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.429173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 176/363] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.429795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.462863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe28ef4-b793-4861-9db9-7e95e96e86fb None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.500645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.501593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.502074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.502568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.503494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] RBAC: Adding query filter params (role.id=7bf924eb503449ccaddc82aad1e450af, user.id=79d02b4e3d074594aab27bc1319938e2, scope.system=all) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:43.503999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.505706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.524960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 188/364] 10.222.0.27 () {56 vars in 1308 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/role_assignments?user.id=79d02b4e3d074594aab27bc1319938e2&scope.system=all&role.id=7bf924eb503449ccaddc82aad1e450af => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.525256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.525340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.599843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a9213-8d6a-4438-b9e7-cf71b44c6dd9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.629528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.655994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.656662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.656943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.657312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.658442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.659985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.662059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 177/365] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.662930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.685861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9161aca-3fe9-4aba-aa13-9b26ee72bab7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.732563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.733283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.733523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.734380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.735779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:43.737353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:43.740063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 189/366] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:43.740063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.806458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253845ef-efc8-4a53-9caf-f09b710e75ad None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.836258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.851265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.852632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.852632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.852824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.865193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:43.871124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:43.872026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:43.872884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 178/367] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:43.873558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:43.880999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff8cd-8e16-44e2-b1d5-6d76b5a216f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:43.905585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:43.924061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:43.925042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:43.925324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:43.925598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:43.939784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:43.945812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:43.946716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:43.947904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 190/368] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:43.948842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.006650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee302f87-a6dc-44c1-82cd-b73b4a65ddd7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.050933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.051381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.051653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.052100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.053075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=e9483c5eaf14474197c855467cfe5a7d) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:44.053993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.055441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.063159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 179/369] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=d53450bb086544ce968b603c0997c5b9 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:44.063159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.082846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5664e2-68fa-4bd6-839b-84d717c08f0a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.127482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.130356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.132211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.132762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.133210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.135015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:44.135609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.137568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.145273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 191/370] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:43 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=3fbe9323075347109c2f57834701d0de => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:44.146547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.224025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.243140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b0a24-e229-4c59-9d00-078fe0fb0385 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.318029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.319244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.319765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.320767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.321988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:44.322790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.325259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.326326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0b73c-3f57-45f6-8759-f6f3a161dd71 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.343460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 180/371] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:44.349542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.395910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.396536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.396858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.397706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.398375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:44.398671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.400421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.410786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 192/372] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:44.418670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.477750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa809a-2579-4822-856b-cc477cd9469d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.488440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.543122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.543744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.544020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.544286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.560398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cb3ad-2106-4158-8b83-5f3a3f0e2064 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:44.578071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.627838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:44.628456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.628815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.629058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.674974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.676761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7cdfdd-902c-4603-86ff-5ad70eecf944 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.723981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:44.726300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889aaa65-e320-4dd7-be9e-a6b3e5cd4994 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:44.807706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 181/373] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:44.809509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea64f-f872-48e2-ae81-635368244bff None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.809807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea64f-f872-48e2-ae81-635368244bff None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.810089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea64f-f872-48e2-ae81-635368244bff None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.811345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 182/374] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:44.817550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eea64f-f872-48e2-ae81-635368244bff None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:44.831228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.866246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 193/375] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:44.867593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:44.868639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:44.868935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:44.892519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:44.903975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:44.906658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:44.947715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eea64f-f872-48e2-ae81-635368244bff None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.000115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.000876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.001062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.001325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.002454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.004005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.010619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 183/377] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:44 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:45.010619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.010878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 194/377] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:45.014431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.095918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:45.120090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c68db-d7e6-42b9-b5a4-e794966d54a0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.144982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cfe5c-19d2-470e-b37f-eecb2ad29e4c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.157866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.158502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.158776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.159041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] PATH_INFO: `/v3/projects/612c023f1ed44b9f936be9c752c292ec` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.160219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] RBAC: Authorizing `identity:get_project(project_id=612c023f1ed44b9f936be9c752c292ec)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.161671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.163578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 184/378] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/projects/612c023f1ed44b9f936be9c752c292ec => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:45.168851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.192842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.193883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.194878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.195351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.196698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.198381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.204724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 195/379] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:45.206637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.210638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:45.279687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59342a-ea48-4416-a520-1c7a04f90cfb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.317943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.318607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.318872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.319145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] PATH_INFO: `/v3/groups/fca537ddae5448cd9cbd086c39969562` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.323139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd5dc55-e10e-4378-bf52-e4855d312b9d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.327669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] RBAC: Authorizing `identity:get_group(group_id=fca537ddae5448cd9cbd086c39969562)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.328923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.330861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 185/380] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/groups/fca537ddae5448cd9cbd086c39969562 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:45.336509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.363060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.364006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.364506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.365002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.366471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:45.366999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.368695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.372964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:45.380763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 196/381] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=d53450bb086544ce968b603c0997c5b9 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:45.451346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292be00-bddb-49d3-a1b8-d23e8ed8f8e0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.490326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.491554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.491830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.492092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.492853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] RBAC: Adding query filter params (group.id=fca537ddae5448cd9cbd086c39969562, role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=612c023f1ed44b9f936be9c752c292ec) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:45.493152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.494608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:45.501163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 186/382] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/role_assignments?group.id=fca537ddae5448cd9cbd086c39969562&scope.project.id=612c023f1ed44b9f936be9c752c292ec&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:45.585767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.706437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:45.750356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c47a-276a-4319-a782-e3330a2f5d94 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.801879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.802527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.802806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.803081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.816537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:45.821870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:45.822762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:45.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 197/383] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:45.828766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:45.926781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:45.959326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c276cc-0f48-4fc1-9e13-188486dee310 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:45.995702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:45.996294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:45.996569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:45.996908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:45.997572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:45.997871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:45.999236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:46.008883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 187/384] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:46.017684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:46.159858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaf285-7c76-45b2-9a3f-259726fd3c6d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:46.189025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:46.203113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:46.203771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:46.204049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:46.204323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:46.205486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:46.207007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:46.208755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 198/385] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:46 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:46.216002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:46.353764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78894-ce98-4c60-ac10-4aecd44bb76a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:46.395264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:46.395905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:46.396180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:46.396449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:46.397649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:46.399088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88f8da-a128-4696-a55f-4f9d6440d435 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:46.401298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 188/386] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:46 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:46.406931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:46.587692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263bdcd-a3ac-4c69-b3f4-4729516df7f9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:46.643225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:46.644282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:46.644837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:46.645322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/groups/242b62509d0449b09801fe44715864c8/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:46.651654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, group_id=242b62509d0449b09801fe44715864c8, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:46.653422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be24e-2cd8-431b-a4ba-345ff278feb0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:46.671192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 199/387] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Sep 20 04:38:46 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/groups/242b62509d0449b09801fe44715864c8/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:47.274292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695541d6-8d00-47df-99fe-e97da6876aed None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:47.274931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695541d6-8d00-47df-99fe-e97da6876aed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:47.275115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695541d6-8d00-47df-99fe-e97da6876aed None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:47.276721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 189/388] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:47.281814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:47.282268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:47.282347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:47.309548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:47.310794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:47.326611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1699b9-8d19-4e5e-9f5c-87e67461b4fd None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:47.327639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1699b9-8d19-4e5e-9f5c-87e67461b4fd None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:47.328470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1699b9-8d19-4e5e-9f5c-87e67461b4fd None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:47.330786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 190/389] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:38:47.337474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:47.338926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:47.339893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:47.349341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:47.374218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:47.377401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:47.395101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:47.427184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 200/390] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:47.486598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 191/391] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:47.811514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:47.923142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742faa0-c883-4620-9445-9b51d7b36e3d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:47.924105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:47.976309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:47.977349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:47.977858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:47.978341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:47.989177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:47.994814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:47.995864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:47.996772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:47.999034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 201/392] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:48.008537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.063063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.105390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbbcf0-ba44-4962-aa10-e37faded34f9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.120530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56087439-e953-47c3-801d-6a899ddee891 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.147729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.153342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.154424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.154820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.155203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.166069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:48.169631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.170240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.170515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.170785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e383cb6b-1322-4769-a93d-ab7ac77ffecc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:48.171034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:48.171895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:48.177602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:48.178274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:48.178274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.178413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 192/393] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:48.178487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9433cf3-7f20-4728-ac09-0aa1f581fd05 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13d99-cad7-4e69-86f5-c368fec3a4e3 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13d99-cad7-4e69-86f5-c368fec3a4e3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae13d99-cad7-4e69-86f5-c368fec3a4e3 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 193/394] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786f9a1-6d6f-49e0-8d20-27b1a66cc3d1 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.178562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786f9a1-6d6f-49e0-8d20-27b1a66cc3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.179805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786f9a1-6d6f-49e0-8d20-27b1a66cc3d1 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.179805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 194/395] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:48.181439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786f9a1-6d6f-49e0-8d20-27b1a66cc3d1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.190030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 202/396] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:48.192014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054242dc-582b-460b-b5c0-85a1a2a5602e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.192953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054242dc-582b-460b-b5c0-85a1a2a5602e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.193437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054242dc-582b-460b-b5c0-85a1a2a5602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.213033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-054242dc-582b-460b-b5c0-85a1a2a5602e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:48.214221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-054242dc-582b-460b-b5c0-85a1a2a5602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:48.284027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 203/397] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:48.285331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.285622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.285895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.299514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786f9a1-6d6f-49e0-8d20-27b1a66cc3d1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.310257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:48.311765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:48.338498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.339042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.339315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.339597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.340268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] RBAC: Adding query filter params (name=member) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:48.340572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.342077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.349112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.355007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 195/398] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:48.355940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.408770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 204/399] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:48.408770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.423837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.508909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af57ae4-a5a2-4324-8ac0-7b1d11bd06ad None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.570194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.570830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.571108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.571382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.573008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.574447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.576010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 196/400] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:48.578208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a20765-902b-4753-a6f7-237e5df868b7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.581457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.623993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.624539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.625152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.625451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.627165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.628733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.630731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 205/401] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:48.631577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.675169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.720882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ad23-8d22-4266-bba4-040b19119964 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.748269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.748431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b4941-63b0-4309-bcbb-f69fb31730f8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.762007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.762637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.762918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.763189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.764309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.765778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.767540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 197/402] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:48.768137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.786390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.787153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.787471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.787747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.799224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:48.806961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:48.808075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:48.808944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 206/403] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:48.809653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.889091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d70c2-4bc0-4b18-bab7-0531205d608a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.912547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:48.920143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70474645-aad8-4ef5-aa77-b3955e5b6e7b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:48.924500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.925181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.925478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.925731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.926879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:48.928868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:48.935305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 198/404] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:48.935305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:48.961200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:48.961934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:48.962211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:48.962599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:48.973346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:48.979019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:48.979898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:48.980739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 207/405] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:48.981402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.000406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.044582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a97118-6c29-45cc-a555-9096e89d769c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.079558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.080223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.080507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.080816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d/groups/242b62509d0449b09801fe44715864c8/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.083247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9483c5eaf14474197c855467cfe5a7d, group_id=242b62509d0449b09801fe44715864c8, role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.084883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.091449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40e486-5c3f-4a42-8f57-229c62177c6a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.100516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 199/406] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Sep 20 04:38:48 2022] PUT /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d/groups/242b62509d0449b09801fe44715864c8/roles/d53450bb086544ce968b603c0997c5b9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:49.101410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.154741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.156976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.157798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.158289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.160248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:49.160794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.162405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.176132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 208/407] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:49.178800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.206276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.238643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76257-4a3b-43dd-9722-6424a3fa7e74 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.259612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.283417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.284430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.284883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.285316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/groups/242b62509d0449b09801fe44715864c8/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.288141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, group_id=242b62509d0449b09801fe44715864c8, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.289633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.312627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 200/408] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Sep 20 04:38:49 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/groups/242b62509d0449b09801fe44715864c8/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:49.312627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.322056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfdf28-4211-4ea1-b601-2fb510b826c8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.378201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.378839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.379116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.379387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.382955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:49.383260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.387680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.417236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 209/409] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:49.433756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.511299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d1dbe-4ffc-4a20-a824-5fcbbc5f392b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.537307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.554031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.554930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.555210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.555647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] PATH_INFO: `/v3/groups/fca537ddae5448cd9cbd086c39969562` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.557358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] RBAC: Authorizing `identity:get_group(group_id=fca537ddae5448cd9cbd086c39969562)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.558785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c4495-045b-4e8c-b697-72f5f0ef73d5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.558912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709060b-1ccf-46d7-ba59-9f8a8587c9e5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.561714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 201/410] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:49 2022] GET /identity/v3/groups/fca537ddae5448cd9cbd086c39969562 => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:49.562866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc52d-02b3-44d9-9919-a0683765a7ab None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.563144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc52d-02b3-44d9-9919-a0683765a7ab None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.563416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdc52d-02b3-44d9-9919-a0683765a7ab None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.564626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 202/411] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:49.573463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdc52d-02b3-44d9-9919-a0683765a7ab None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.605259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.606315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.606803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.607452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.609455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.611468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ce94a-1bca-4c78-8efb-fc3d52a86d6e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.613594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 210/412] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:49 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:49.614958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.615460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.615935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.627672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.637225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:49.638661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:49.681814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdc52d-02b3-44d9-9919-a0683765a7ab None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.718601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.719119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 211/413] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:49.720214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.725637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.726032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.726400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] PATH_INFO: `/v3/projects/612c023f1ed44b9f936be9c752c292ec` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.729363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] RBAC: Authorizing `identity:get_project(project_id=612c023f1ed44b9f936be9c752c292ec)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.730983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.737495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 203/414] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:49 2022] GET /identity/v3/projects/612c023f1ed44b9f936be9c752c292ec => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:49.738791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.813110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:49.837822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c50c9-e564-4401-a9f7-0f189883d4b3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:49.876849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:49.877804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:49.878270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:49.878730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:49.880016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:49.881673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:49.883552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 212/415] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:49 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:49.889345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:49.979764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201d5cb-a748-49fc-98d0-00dc05198341 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:50.011934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:50.055429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:50.056105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:50.056380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:50.056667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] PATH_INFO: `/v3/projects/612c023f1ed44b9f936be9c752c292ec/groups/fca537ddae5448cd9cbd086c39969562/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:50.059703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=612c023f1ed44b9f936be9c752c292ec, group_id=fca537ddae5448cd9cbd086c39969562, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:50.061945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:50.073608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26005264-e6f4-4f8c-8d47-5b7005fe59b7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:50.083950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 204/416] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Sep 20 04:38:49 2022] PUT /identity/v3/projects/612c023f1ed44b9f936be9c752c292ec/groups/fca537ddae5448cd9cbd086c39969562/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:38:50.085403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:50.213997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:50.214685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:50.214962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:50.215233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/groups/242b62509d0449b09801fe44715864c8/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:50.222645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa845cee-67cd-4751-a750-3c3be5ab81a0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:50.226704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef5b5015f01d4696a85ee985135fe7d6, group_id=242b62509d0449b09801fe44715864c8, role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:50.228165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:50.267692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 213/417] 10.222.0.27 () {58 vars in 1442 bytes} [Tue Sep 20 04:38:49 2022] PUT /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6/groups/242b62509d0449b09801fe44715864c8/roles/d53450bb086544ce968b603c0997c5b9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:38:50.271462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:50.272510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:50.273070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:50.273641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:50.287346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:50.293643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:50.294640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:50.295533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:50.298441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 205/418] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:50.305268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:50.318281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:50.487557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d8d8a-1fbd-43cf-a236-6419cafc8c18 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:50.522957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:50.523488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:50.523762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:50.524026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:50.524947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:50.525249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:50.526686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:50.535399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 214/419] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:50.541556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:50.581960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:50.662073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d61b7-5f45-45e4-85ac-74066f2638a3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:50.697573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:50.698589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:50.699067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:50.699557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:50.774283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:50.776375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:50.811428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:50.908535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 206/420] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:50.908758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:50.996211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:51.034898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc0c5-f50f-4216-acc7-b3d5455939aa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:51.069699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:51.070394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:51.070674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:51.070954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:51.072154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:51.073632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:51.075533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 215/421] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:50 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:51.081106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:51.190817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3ef55-ad70-400f-8d7c-69f56c3bfad7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:51.227694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:51.228342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:51.228639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:51.228918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:51.230127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:51.231576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41828d2e-d633-41d5-bf0d-6c3775fe0d7a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:51.233591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 207/422] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:51 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:51.239491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:51.351438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba2ea7-f39f-4813-ba92-3b7df7fe30e9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:51.385764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:51.386370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:51.386648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:51.386922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:51.387638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=e9483c5eaf14474197c855467cfe5a7d) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:51.387943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:51.389398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306bfb9-8b12-4b00-a46b-3d975e05d669 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:51.392276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:51.400009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 216/423] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:51 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=3fbe9323075347109c2f57834701d0de => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:51.659792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad79a7-4e3d-440f-8780-451b4400830c None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:51.660298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad79a7-4e3d-440f-8780-451b4400830c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:51.660298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad79a7-4e3d-440f-8780-451b4400830c None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:51.661644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 208/424] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:51.669250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d240-d109-47af-96e3-c6ed48fb7a17 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:51.669722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d240-d109-47af-96e3-c6ed48fb7a17 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:51.669801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d240-d109-47af-96e3-c6ed48fb7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:51.690372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ded1d240-d109-47af-96e3-c6ed48fb7a17 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:51.691578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ded1d240-d109-47af-96e3-c6ed48fb7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:51.768722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 217/425] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:52.045338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad79a7-4e3d-440f-8780-451b4400830c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.048282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:52.056853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f923-2044-413d-b8df-b5d2492dcde2 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.057135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f923-2044-413d-b8df-b5d2492dcde2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.057409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1f923-2044-413d-b8df-b5d2492dcde2 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.058870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 218/426] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:52.065302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.065579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.065848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.091972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:52.093570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:52.185086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad79a7-4e3d-440f-8780-451b4400830c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:52.200641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 219/427] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:38:52.235210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:52.235955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.236298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.236654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.247687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:52.252788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:52.253696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:52.254661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 209/428] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:52.261853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.331014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:52.377910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ef6d9-7cb9-4045-a1b8-3dacde1c768d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:52.416161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:52.416798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.417077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.417345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.418009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:52.418310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:52.419671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:52.431805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 220/429] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:52.437718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.519979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:52.541849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.566228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62d6b-0b57-4ba0-9590-b447f415a982 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:52.615094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:52.615661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.615944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.616220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.685007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bda922-caff-4bbd-99d8-e89b320885a0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:52.698149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:52.699703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a7005-78ae-4890-9791-a592ea2aa35d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:52.712630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:52.726222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:52.726855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.727133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.727396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.738482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-5fc08940-d608-44b0-aaa0-db4232fed6dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:52.743705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:52.744650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 221/430] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:52.750827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1081df-ef2a-45f3-8422-4fdb9ff70b8c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.752145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1081df-ef2a-45f3-8422-4fdb9ff70b8c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.752145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1081df-ef2a-45f3-8422-4fdb9ff70b8c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.752145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 222/431] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:52.752623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1081df-ef2a-45f3-8422-4fdb9ff70b8c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.773554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:52.815561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 210/432] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:52.817148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.817434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.817710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.837339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:52.838519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:52.872042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1081df-ef2a-45f3-8422-4fdb9ff70b8c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:52.912228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:52.913044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 211/433] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:52.913668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.913998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:52.914431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:52.914925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:52.916144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:52.916637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:52.920205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:52.931845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 223/434] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:52.937625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:52.987979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.025091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.033981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12940dd1-1baf-43ce-a1b6-cbb940d32419 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.049404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeddf4-c53e-4a63-9166-64d75b1460c6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.068863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.069500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.069781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.070054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] PATH_INFO: `/v3/projects/e9483c5eaf14474197c855467cfe5a7d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.071181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] RBAC: Authorizing `identity:get_project(project_id=e9483c5eaf14474197c855467cfe5a7d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.072658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282de36-bf19-4762-894c-0fbcd12b7e25 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.078571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 212/435] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/projects/e9483c5eaf14474197c855467cfe5a7d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.078571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca990-6d52-45dd-b9bc-ce2bb5f40085 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.078571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca990-6d52-45dd-b9bc-ce2bb5f40085 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.078571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca990-6d52-45dd-b9bc-ce2bb5f40085 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.078571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 213/436] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:53.082238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca990-6d52-45dd-b9bc-ce2bb5f40085 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.090272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.090881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.091158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.091678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.170067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.171788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.196099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca990-6d52-45dd-b9bc-ce2bb5f40085 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.219788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.230903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.231528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.231796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.232061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.233174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.234599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05cb4b3-ecc3-4aec-8bd1-12e9e29131ba None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.236139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 214/437] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.241615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.241615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.241615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.255280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:53.257089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:53.278929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.280681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 224/438] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 20 04:38:53.281503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.322897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 215/439] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:53.323725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.385193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b67f4f-a8ab-4500-a1e5-6498da3146a5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.420983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.421537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.421818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.422090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.426428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5959366-da38-4b9e-800c-1794dbe87102 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.433828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:53.438808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:53.439798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:38:53.440618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:53.442990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 225/440] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:53.444460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.449292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.470130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.471047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.471589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.471863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.473852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=e9483c5eaf14474197c855467cfe5a7d) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:53.474162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.477693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.496639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 216/441] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=e9483c5eaf14474197c855467cfe5a7d&role.id=d53450bb086544ce968b603c0997c5b9 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.496639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.529627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.566806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d71a0-1d90-48cd-aa6e-96908bbd17d6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.612588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.613244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.613519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.613783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.615059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.617229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.618900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 226/442] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.621056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.629824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea8da8-f6a6-420e-bda9-038ea865c7ac None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.669251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.669858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.670135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.670403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.671801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:53.671801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.672770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.682726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 217/443] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.683545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.704687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.740681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa72e6-dd5a-40cf-bfb8-ac9be85a5d63 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.778756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.779419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.779697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.779969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.789787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.790836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:38:53.795618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:53.796551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:38:53.797504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:53.798132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 227/444] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:53.802373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.812802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cc25b-6246-4e27-9123-fe835cea2277 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.855180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.855813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.856096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.856372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.857537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:53.858978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:53.860537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 218/445] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:53.863802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:53.916664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bc66b-03e2-4858-9ae1-beb784bd8b83 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.944384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:53.950085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:53.950686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:53.950968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:53.951238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:53.964620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e78a77-e17b-4a04-bb0c-efd6c036e0ea None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:53.999882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.000487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.000797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.001073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.001731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:54.002034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.003410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.011111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 219/446] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:54.011888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.022618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.024128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.027720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.110186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74cb8b-1467-439f-abe9-47cc0dae53de None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.121470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 228/447] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:54.122302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.143442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.144032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.144313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.144581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.145328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=3fbe9323075347109c2f57834701d0de, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:54.145634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.147565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.153642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 220/448] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=3fbe9323075347109c2f57834701d0de => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:54.154345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.156104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.241860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f57f5e-f47b-4ae4-9ec8-ebfc7447efe3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.245550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.270177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c042c-7ad9-4426-8efd-31f4a88da4cc None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.283671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.284324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.284649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.284901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.306966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.307613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.307893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.308164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] PATH_INFO: `/v3/projects/612c023f1ed44b9f936be9c752c292ec` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.309436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=612c023f1ed44b9f936be9c752c292ec)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.310924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.312991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 221/449] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/projects/612c023f1ed44b9f936be9c752c292ec => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:54.318168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.372046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.373647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.414835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.433856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074f078-d0ea-48ce-9924-5bc919e77ff2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.471966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.473019 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.473019 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.473241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] PATH_INFO: `/v3/groups/fca537ddae5448cd9cbd086c39969562` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.474384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] RBAC: Authorizing `identity:get_group(group_id=fca537ddae5448cd9cbd086c39969562)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.476349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.477938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 222/450] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/groups/fca537ddae5448cd9cbd086c39969562 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:54.483841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.502220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.507208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 229/451] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:38:54.508495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.595856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52fe3-e231-48ae-b0b9-ff362e251790 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.613569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60266bf-8b61-48bf-aa2f-fed4dab5af08 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.630728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.631348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.631624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.631894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.632674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] RBAC: Adding query filter params (group.id=fca537ddae5448cd9cbd086c39969562, role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=612c023f1ed44b9f936be9c752c292ec) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:54.633002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.634422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.639899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 223/452] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/role_assignments?group.id=fca537ddae5448cd9cbd086c39969562&scope.project.id=612c023f1ed44b9f936be9c752c292ec&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:54.642586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.648476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.649157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.649438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.650001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] PATH_INFO: `/v3/projects/ef5b5015f01d4696a85ee985135fe7d6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.651173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] RBAC: Authorizing `identity:get_project(project_id=ef5b5015f01d4696a85ee985135fe7d6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.652683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.654595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 230/453] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/projects/ef5b5015f01d4696a85ee985135fe7d6 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:38:54.660206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.747154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:54.768952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5930cb-7e68-47ab-bb11-9d795382a549 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.804379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.805091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.805304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.805572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] PATH_INFO: `/v3/groups/242b62509d0449b09801fe44715864c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.806681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] RBAC: Authorizing `identity:get_group(group_id=242b62509d0449b09801fe44715864c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.808125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e3e8-7095-4ac6-85c6-d72ae856a4f9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.809702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 224/454] 10.222.0.27 () {56 vars in 1180 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/groups/242b62509d0449b09801fe44715864c8 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:54.815669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:54.913748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69d08e-af15-46c2-a14d-d7a2786eca59 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:54.945727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:54.949804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:54.950110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:54.950382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:54.951110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] RBAC: Adding query filter params (group.id=242b62509d0449b09801fe44715864c8, role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=ef5b5015f01d4696a85ee985135fe7d6) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:54.952298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:54.953774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7811235-4b6c-462f-ab20-9898ae42e4b3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:54.960158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 231/455] 10.222.0.27 () {56 vars in 1376 bytes} [Tue Sep 20 04:38:54 2022] GET /identity/v3/role_assignments?group.id=242b62509d0449b09801fe44715864c8&scope.project.id=ef5b5015f01d4696a85ee985135fe7d6&role.id=d53450bb086544ce968b603c0997c5b9 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:57.743718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90c39-0abb-4c55-a5a2-287fe32d9575 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:57.744236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90c39-0abb-4c55-a5a2-287fe32d9575 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:57.744236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd90c39-0abb-4c55-a5a2-287fe32d9575 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:57.745875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 225/456] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:57.751135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d737cb-8302-4933-9063-6fad86ca5fd2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:57.751558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d737cb-8302-4933-9063-6fad86ca5fd2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:57.751806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d737cb-8302-4933-9063-6fad86ca5fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:57.776563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-66d737cb-8302-4933-9063-6fad86ca5fd2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:57.777823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-66d737cb-8302-4933-9063-6fad86ca5fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:57.862879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:57.871952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 232/457] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:57.915390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bae6-4114-430a-9972-33cfc9e25d67 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:57.916002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bae6-4114-430a-9972-33cfc9e25d67 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:57.916496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93bae6-4114-430a-9972-33cfc9e25d67 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:57.918307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 226/458] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:57.923284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94797-ca6e-4a33-a94c-9c1dfd958d20 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:57.923574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94797-ca6e-4a33-a94c-9c1dfd958d20 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:57.923849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94797-ca6e-4a33-a94c-9c1dfd958d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:57.959129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-74b94797-ca6e-4a33-a94c-9c1dfd958d20 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:57.959129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-74b94797-ca6e-4a33-a94c-9c1dfd958d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:58.055020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 233/459] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:58.233795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9424f-9d12-48ef-be2c-1560e95b06f6 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.234452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9424f-9d12-48ef-be2c-1560e95b06f6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.234937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9424f-9d12-48ef-be2c-1560e95b06f6 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.236862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 227/460] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:58.257463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.258109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.258578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.285596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:58.287120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:58.324459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9424f-9d12-48ef-be2c-1560e95b06f6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.369762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 234/461] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:58.370963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.429021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:58.463162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9424f-9d12-48ef-be2c-1560e95b06f6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.475799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d8186-d037-4b9a-acf9-70970ebc6c3f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.499769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:58.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.501191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.501654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.512772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:58.513658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.514153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.514626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.515431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:58.517036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:58.524046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:58.526280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:58.527954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ccf4f8c7-70b7-4ed0-af0d-ad2ac1d27062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:38:58.534560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:58.535794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:38:58.536974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:38:58.536974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.536974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:58.543972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 228/462] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:58.545429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06e4ea-28ac-42d5-9182-dd551a177e8f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:38:58.545536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:58.546399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:38:58.547256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:38:58.547256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.547256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:38:58.547256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 235/463] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:58.548075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06e4ea-28ac-42d5-9182-dd551a177e8f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.548506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06e4ea-28ac-42d5-9182-dd551a177e8f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.550197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 229/464] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:58.552529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.553848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06e4ea-28ac-42d5-9182-dd551a177e8f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.660148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff213f-4253-492a-84ed-aed8dcd3ac9d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.667197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:58.668259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06e4ea-28ac-42d5-9182-dd551a177e8f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.697934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:58.698906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.699343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.699620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.700252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:58.700553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:58.701943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243d8c0-623f-442a-9de9-3a8d9d5ffc05 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:58.704022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:58.704648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.704934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.705210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.705845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:58.706177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:58.707571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:58.715373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 236/465] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:58.716906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.717356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.717771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.731075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 230/466] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:58.732297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.736537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:58.737755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:58.796483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:58.813761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 237/467] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:58.815078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.852634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60f80a-6f4e-4ff1-bc62-c2cfc18e410f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.889422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:58.890184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:58.890536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:58.890873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:58.903061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:58.904802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:38:58.921493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:58.922411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:38:58.923305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:38:58.923305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:58.923305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:58.923305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 231/468] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:58.923305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:58.946132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5404db-d801-4bdd-abf9-b53131a91007 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:58.969962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:59.001308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.002269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.002720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.003143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.003948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.005629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.032221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 238/469] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:58 2022] POST /identity/v3/users => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:59.034079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.062837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d14cf6-ac06-4250-8b66-00af7255e731 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.086867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:59.144188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.144850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.145138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.145412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.146084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.148204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.182595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 232/470] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:58 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:59.197301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.200997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d961ac4-0981-425b-8e7a-01c04ac3e1f7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.283154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.285022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.285305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.285577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.286244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:59.286551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.289648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.312277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 239/471] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:59.323035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.342340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:59.373839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34266d-e791-4313-9da9-8550367f920c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.414952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.415634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.415916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.416243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.429251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.430749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.441469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e47-f925-4fc6-b791-fa75bec85478 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d2d7c50f-322c-49aa-8eb5-0d452eae909b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:38:59.448309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:38:59.449329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:38:59.450196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:38:59.450196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:38:59.450196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:38:59.450196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 233/472] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:38:59.451014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f1b6-3251-4fe4-b4d1-6e36e071827e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.451297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f1b6-3251-4fe4-b4d1-6e36e071827e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.451569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10f1b6-3251-4fe4-b4d1-6e36e071827e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.456629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 234/473] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:38:59.459052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.459863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.460141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.480136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:38:59.481391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:38:59.489731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.490416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.491237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.491877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.492586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.494412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.495682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:59.518933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 240/474] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:59 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:38:59.526895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.573237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 235/475] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:38:59.622231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:38:59.664982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa080752-b20d-4a5a-8def-a020522bb289 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.702881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.703830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.704264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.704713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.705515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:38:59.706021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.707661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.723062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 241/476] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:38:59.728722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.865411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c8603-2c73-4cb2-88d7-a37d483ccca4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:38:59.872091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:38:59.912799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:38:59.913448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:38:59.913732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:38:59.914001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:38:59.914726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:38:59.916359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:38:59.951690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 236/477] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:38:59 2022] POST /identity/v3/users => generated 249 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:00.010491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515247-8134-4669-85ab-b84a091d487e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:00.050015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:00.059128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:00.059752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:00.060026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:00.060290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:00.061027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:00.062434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:00.084495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.sql.core [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 20 04:39:00.084495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 20 04:39:00.084495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [parameters: {'id': 'cb53dd1b92024ff788b906414a29d920', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 20 04:39:00.084495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87797) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:00.101693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return ref.to_dict() Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:00.102656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application yield resource Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.session.rollback() Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.force_reraise() Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise self.value Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application session.commit() Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 20 04:39:00.103591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._prepare_impl() Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.session.flush() Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._flush(objects) Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application compat.raise_( Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application flush_context.execute() Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rec.execute(self) Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 20 04:39:00.104476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:00.105569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:00.106614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [parameters: {'id': 'cb53dd1b92024ff788b906414a29d920', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:00.107615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 20 04:39:00.108661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:00.109850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 242/478] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Sep 20 04:38:59 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:00.109932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:00.125538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:00.234204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adcb77-2d38-4995-b961-d06467601045 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:00.270245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:00.271172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:00.271651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:00.272106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:00.284563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-df6caca8-4d39-4a7a-b6f9-0c8fc99817fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:00.291490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:00.292652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 20 04:39:00.293644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:00.295998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 237/479] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:00.303703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:00.427780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe7e-2aea-44d6-aee0-1eab04f510a4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:00.463708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:00.464346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:00.464623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:00.464900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:00.465568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:00.465870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:00.467242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a1607-3b52-4fb0-9390-be0980b741c1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:00.480736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 243/480] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:01.743361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865977-9a1e-454d-aa71-95d959712fe4 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:01.743883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865977-9a1e-454d-aa71-95d959712fe4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:01.743883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865977-9a1e-454d-aa71-95d959712fe4 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:01.745534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 238/481] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:01.750833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8613e08-1589-44f5-beeb-064d58441fc9 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:01.751252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8613e08-1589-44f5-beeb-064d58441fc9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:01.751443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8613e08-1589-44f5-beeb-064d58441fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:01.772969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b8613e08-1589-44f5-beeb-064d58441fc9 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:01.774174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b8613e08-1589-44f5-beeb-064d58441fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:01.842672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:01.867255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 244/482] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:02.094054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9c4e1-3144-4a37-9a26-3fd22cde2efa None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.094389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9c4e1-3144-4a37-9a26-3fd22cde2efa None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.094506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9c4e1-3144-4a37-9a26-3fd22cde2efa None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.096112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 239/483] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:02.101720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c878a-bb90-40cf-be5c-35dc7b2dff61 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.102003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c878a-bb90-40cf-be5c-35dc7b2dff61 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.102272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c878a-bb90-40cf-be5c-35dc7b2dff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.125787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f10c878a-bb90-40cf-be5c-35dc7b2dff61 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:02.127010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f10c878a-bb90-40cf-be5c-35dc7b2dff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:02.156837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9c4e1-3144-4a37-9a26-3fd22cde2efa None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.242697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 245/484] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:02.260146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac289f9-b536-4d72-81a8-3da9e02624aa None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.260630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac289f9-b536-4d72-81a8-3da9e02624aa None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.261021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac289f9-b536-4d72-81a8-3da9e02624aa None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.266669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 246/485] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:02.268291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.268707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.269366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.277952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:02.291952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:02.293632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:02.349381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9c4e1-3144-4a37-9a26-3fd22cde2efa None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:02.377729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 247/486] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:39:02.407986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:02.408861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.409280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.409690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.422868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:02.428329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:02.429403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:02.430428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:02.432623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 240/487] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:02.441926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.508723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:02.572849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2bb0e-3324-414f-b3bc-506d246b2bf3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:02.607431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:02.607968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.608247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.608517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.609209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:02.609509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:02.610940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:02.620251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 248/488] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:02.625462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.650410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.770936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1085dec-db29-4d1f-93e8-f6efdc8733bb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:02.778744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:02.809185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4864695-f7f4-4355-94b6-54deace309a6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:02.812574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:02.813444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.813869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.814286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.831858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:02.838769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:02.839692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:02.840531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 249/489] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:02.841319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.857360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:02.871853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:02.872403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:02.872794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:02.873074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:02.888767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:02.890628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:02.913021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:02.913989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:02.914857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:02.914857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:02.914857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:02.916684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 241/490] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:02.917717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:02.986216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5b0c-6a5f-4972-91d0-66890747c3f8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.024105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.025033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.025138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c37a35-475b-4f7a-a1d1-7d4baa5a5e3d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.025317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.025590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.036282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:03.042063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:03.042979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:03.043791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 250/491] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:03.045298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.066181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.067030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.067314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.067588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.068274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.069150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.070539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd16be-2b7e-43c8-a9f0-1697149a5029 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.075254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.079489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 242/492] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.080542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7456-354b-4663-93f0-ca8a0117b10e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.080865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7456-354b-4663-93f0-ca8a0117b10e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.081141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd7456-354b-4663-93f0-ca8a0117b10e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.082482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 243/493] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:03.083115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7456-354b-4663-93f0-ca8a0117b10e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.127526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.162658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8eb8-6ab2-4523-bcf7-9cbc2c24a5d0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.197759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.198369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.198648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.198962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.199546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.199849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.200054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd7456-354b-4663-93f0-ca8a0117b10e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.201245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.219947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 251/494] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.219947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.239455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.240090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.240377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.240676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.241340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.241649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.243087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ff38f-d875-441b-b2e0-8d7dedf54ece None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.256296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 244/495] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.256296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.256296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.256296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.272328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:03.273562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:03.321720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.339313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65c0e-3492-4f34-a101-665e02a366b1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.355010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 245/496] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:03.355769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.386528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.387501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.388739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.389054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.389506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.405275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.407625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:03.428912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:03.430107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:03.431152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:03.431152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:03.431152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:03.432722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 252/497] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:03.432722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.467776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c0b57-9447-4e84-af2c-7b1eddebfdea None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.501144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.501766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.502047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.502303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.509901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.511851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-8aab9b3f-2397-4512-9cfd-99cda73a3f3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:03.525371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:03.526294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:03.527212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 246/498] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:03.527868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484126-e5ba-4256-8ca3-8dd67851fa50 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.527868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484126-e5ba-4256-8ca3-8dd67851fa50 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.532980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9484126-e5ba-4256-8ca3-8dd67851fa50 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.532980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 247/499] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:03.532980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9484126-e5ba-4256-8ca3-8dd67851fa50 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.558009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b1d1-af42-4806-9e85-ea774e30b9f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.594536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.595227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.595704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.595814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.603656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.608194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.609688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:03.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:03.627445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:03.628943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:03.628943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:03.628943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:03.628943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 253/500] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:03.628943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.629688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.644419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9484126-e5ba-4256-8ca3-8dd67851fa50 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.679069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.679661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.679940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.680210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.680874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.681173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.682550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fe7f0-642f-428e-92df-50227af31cdc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.693962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 248/501] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.695178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.695467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.695739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.713835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:03.715133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:03.732120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026110-6920-4e2b-ad7b-95bc92b011a3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.766343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.766946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.767227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.767496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.768215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.768521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.769983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.779737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 249/502] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:03.780557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.808086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 254/503] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.808086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.859249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.870722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:03.890442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee79ec4-b6b1-4d7b-abf1-01212811ff4e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.915317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893dc2-c35e-47e6-ab7c-6e5b6fe7595e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:03.922932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.923558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.923835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.924111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.934771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:03.939526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:03.940395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:03.941226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 250/504] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:03.941891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:03.958893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:03.959730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:03.960186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:03.960448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:03.961128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:03.961429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:03.962796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:03.978782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 255/505] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:03.978782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.053678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0cbc9-cb37-498f-8fe6-b178b08519d6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.084046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec3441-9c79-48d1-bec2-23dcecba46f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.086275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.086889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.087168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.087441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.092634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.094627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.096177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.098800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:04.109651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:04.110631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 251/506] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:04.111574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.121477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.122071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.122349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.122616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.191550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.192954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.210520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203d36-8b77-45fa-93de-df2914a9e8a4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.242273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.242860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.243140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.243410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.244068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:04.244366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.245763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.253371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 252/507] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:04.254152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.290215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 256/508] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:04.291045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.301060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.315784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.354305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259dd97-5d1c-41c1-902c-2a3bb3175d29 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.386829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.387446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.387722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.387989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.390347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d1f3e-f00d-4aaf-bbaf-a97b7180b4f9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.395481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.397543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:04.408458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:04.409377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 253/509] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:04.410273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.427307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.427941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.428216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.428483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.438360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.439874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:04.453854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:04.454951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:04.456091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:04.456091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.456091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:04.456091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 257/510] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:04.456091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.514250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.516133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cae77c-17ad-4b66-a74c-dbf68a34aabc None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.539054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.550789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.551382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.551676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.551930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.552672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:04.552973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.554386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.555063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd94ed-c41d-45df-93cc-e2ff930a8b8d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.587438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.588055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.588334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.588629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.589637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 254/511] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:04.590442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.599179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.601262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:04.615072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:04.616028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:04.617119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:04.617119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.617119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:04.617119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 258/512] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:04.617119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.691746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75475f-edb6-411f-865b-09e069a70f6a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.717351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcef7dd-9c4f-4083-ae58-8a93bafaaabf None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.726245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.726878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.727165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.727439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.729566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.737228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.738683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.750055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.750658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.750940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.751215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.751929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:04.752236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:04.753169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:04.754021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:04.754875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:04.754875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:04.754875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:04.755037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.755111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 255/513] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:04.755111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.776216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 259/514] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:04.777087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.788558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.855499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd0ad9-c72d-4dc0-994a-eeeb0b8ccad6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.873311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461593a9-65c6-4dcb-9d3f-72443812a213 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:04.888554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.889170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.889447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.889717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.890342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:04.890645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.892015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.903537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 256/515] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:04.904306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:04.911389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:04.912025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:04.912306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:04.912582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:04.953246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:04.983361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:04.984778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:04.995449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.005120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5f61-bc89-48ad-a09e-3203c10573ee None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.038932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.039534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.039820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.040088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.040778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:05.041065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.042448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.055768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 257/516] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:05.056580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.096815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 260/517] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:05.097667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.162686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f9a0-5f19-408f-9a68-56d0e35631ef None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.165488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.194974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1cd7f1-170a-4c48-ac6d-454d559d3fbe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.196526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.197154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.197434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.197704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.198335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:05.198635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.200002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.211431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 258/518] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:05.212200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.227914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.229622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.230144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.230417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.230709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.300403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.302184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.311223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7ea87-43ec-41a2-8e39-7406d97e4378 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.343648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.344248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.344527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.344835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.345480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.346896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.377836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 259/519] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:39:05 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:05.377836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.383437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.432685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 261/520] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:05.433769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.458744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.502036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66808a-69f5-44f8-883f-3b00fd89f9be None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.541049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.541693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.541970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.542239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.546657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd859f-f1d0-4ce1-927f-83e1c8e7bd17 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.550606 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.552016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:05.565897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:05.566847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:05.570758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:05.571533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 260/521] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:05.573526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.589259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.590245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.590245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.590499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.601839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.603627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:05.620657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:05.621605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:05.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:05.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:05.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:05.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 262/522] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:05.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.651651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.689460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a838cf0-84d5-4e18-bd28-3032e52ff612 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.719435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.728129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.728978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.729261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.729537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.736142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.737625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.740555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9476e0-8153-4ffb-aefb-76378280fa3c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:05.749750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:05.750653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:05.751506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 261/523] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:05.752023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.787652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.788384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.788698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.788974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.799750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.801308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:05.817049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:05.818029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:05.818939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:05.818939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:05.818939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:05.820665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 263/524] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:05.821646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.863916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9a34d-2cf3-43b1-a4d7-d8b72fdf1bb4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.898646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.899275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.899511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.900046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.900882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:05.901185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.902672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.906745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.926086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 262/525] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:05.926958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:05.934868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483afc5-c82e-4141-8dc2-edee8349f0c2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:05.972170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:05.972821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:05.974214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:05.975125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:05.975419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:05.976127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:05.976437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:05.977812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:05.991284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 264/526] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:05.991925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.031544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407c03f-7842-4721-ae59-9bf5256d2afa None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.065250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.065773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.066048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.066314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.066995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.067297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.069304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.086236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 263/527] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.091099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.100515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e120-d600-407d-9dd2-252a92bf2fd0 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.137255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.137858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.138139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.138413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.139050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.139356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.140760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.142263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.153919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 265/528] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.153919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.203105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b4da-cdbd-4af3-b16c-a4f0a79321f9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.211220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.239102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.239705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.240011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.240288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.267302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6da52-165f-4d4a-a670-8cdcd2b49c5e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.303990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.304694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.304976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.305247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.310793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.312300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.313298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.314759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:06.327522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:06.328429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 266/529] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:06.329357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.391531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.428482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 264/530] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:06.433494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.440628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f4a46-acac-40e7-a122-a0edd83285a8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.441340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.477281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.477952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.478226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.478500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.479204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.479506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.480953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.486745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 267/531] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.487759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.540593 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b50af-a4b5-4f6d-9e81-8e2fce219f04 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.580590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.581195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.581470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.581741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.589788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.591365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:06.604303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:06.605522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:06.606439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 265/532] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:06.606955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.647900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be8403-648b-4327-a766-b5ef7742eb6b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.680981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.687078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.713503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.714504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.714929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.715346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.716241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.716721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.718415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.730643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6dbd-f1a4-4171-9cfa-1aed1a75e07b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.748958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 268/533] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.750096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.770040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.770678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.771937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.772226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.787846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.789326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:06.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:06.809649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:06.810492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:06.810492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:06.810492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:06.816535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 266/534] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:06.817546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.871768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548247e-d39e-4b4f-b91a-f4577c5474bf None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.917648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.918545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.918830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.919109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.920117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.920429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.922308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.925696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ac2b-24e0-4d9f-b853-a0bc208cb719 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:06.936689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 269/535] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.937770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.948845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:06.968218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:06.969200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:06.969817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:06.970266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:06.971162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=ef0eb7d5e5b1416b8018e1a5c49a33e5) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:06.971647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:06.973399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:06.984819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 267/536] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/role_assignments?user.id=ef0eb7d5e5b1416b8018e1a5c49a33e5&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:06.986087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:06.991520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:07.071341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2a970-41c8-44f1-9381-d46bc792db70 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:07.113790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:07.114792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.115152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.116056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.116744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:07.117049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:07.118450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:07.140345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 270/537] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:07.145828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:07.254579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb0275-009a-4d56-8a90-a9d7a1624d5b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:07.264551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9456e-ac36-45c1-9844-2c883c4f7a54 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:07.268985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:07.320418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:07.321078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.321355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.321630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.331154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:07.333040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:07.335237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:07.335847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.336122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.336389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.337148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=f7cdae17359d4ba2ae149afbbd855ee7) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:07.337456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:07.338886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:07.354134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 268/538] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:06 2022] GET /identity/v3/role_assignments?user.id=f7cdae17359d4ba2ae149afbbd855ee7&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:07.356094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:07.356963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:07.357826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 271/539] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:07.364366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:07.444294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:07.532061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381aaa3-e36e-4f2c-81cd-2d20c22774cc None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:07.586268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:07.586880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.587159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.587433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.589122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:07.589122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:07.590003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3078b-62d8-4dff-af43-f81a203e323f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:07.609107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 269/540] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:07.614965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:07.735069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c3870-ae51-4d49-96b9-2546069a6913 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:07.786022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:07.786636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.787428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.787699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.788422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=2fe2203ea1f94623956a215190e4d79c) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:07.788754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:07.790194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae774c-e198-42e2-bbc2-596feb71db7a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:07.800948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 272/541] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:07 2022] GET /identity/v3/role_assignments?user.id=2fe2203ea1f94623956a215190e4d79c&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:07.817548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c6ca3-9b85-42e5-9760-b4b6fea46245 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.817867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c6ca3-9b85-42e5-9760-b4b6fea46245 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.818133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c6ca3-9b85-42e5-9760-b4b6fea46245 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.819477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 270/542] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:07.824987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:07.825275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:07.825554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:07.844811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:07.846063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:07.865799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:07.946885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 273/543] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:08.308403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c6ca3-9b85-42e5-9760-b4b6fea46245 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:08.437375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c6ca3-9b85-42e5-9760-b4b6fea46245 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:08.445636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:08.480088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:08.480746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:08.481030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:08.481304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:08.500303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:08.505133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:08.506085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:08.506905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:08.507542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 271/544] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:08.514478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:08.662599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7330a-22ce-409c-a57e-92c6f5c4c499 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:08.708653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:08.710226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:08.710685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:08.711153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:08.712131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:08.712386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:08.713788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575fb2b-6ada-4109-a3e3-51305ee33ae8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:08.723059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 274/545] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:08.728563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:08.894050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8188ef-df80-4ea6-8717-cf45e3241049 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:08.928125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48b91-51ce-4fa3-9761-f0b5ec8fd19a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:08.928444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48b91-51ce-4fa3-9761-f0b5ec8fd19a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:08.928741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c48b91-51ce-4fa3-9761-f0b5ec8fd19a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:08.930196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 275/546] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:08.934788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:08.935326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:08.935942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:08.945304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:08.950164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:08.950667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:08.951141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:08.966265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:08.969516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:08.970675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:08.974169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:08.989640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:09.004863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:09.006425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:09.007275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:09.007275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:09.007275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:09.009838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 272/547] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:09.017512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.050755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.069100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 276/548] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:09.187249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca8150-17d0-4179-b2bd-c995e4ce985d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.223649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.224690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.225186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.225656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.226869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:09.227384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:09.229256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:09.242025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 273/549] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:09.248630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.397350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf23613-8953-45ef-890a-d9b37959f815 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.407814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.447830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.448465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.448767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.449039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.459177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:09.460769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:09.491579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:09.493159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:09.494204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 277/550] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:09.498679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.520626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.520834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.557582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbfcdff-3eb3-4b84-a6f0-2f34e4efea5b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.597790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.598782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.599292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.600438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.611295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:09.618519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:09.619450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:09.620454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:09.623047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 274/551] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:09.633505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.646851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3964b-86a7-452f-812c-e5ae448bfa4e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.717983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.718722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.718924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.719195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.719932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:09.720235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:09.721815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:09.754045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa3cdd-bf1a-49ac-9312-fd34d21eea84 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.765618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 278/552] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:09.771748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.820021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.820650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.820939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.821213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.821876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:09.822181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:09.823578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378d4c3-de78-426c-a32d-771e3d868baf None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:09.828852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.833157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 275/553] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:09.835145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c20db-e486-49fc-93bc-0be35c95937e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.835934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c20db-e486-49fc-93bc-0be35c95937e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.836707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c20db-e486-49fc-93bc-0be35c95937e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.838385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 276/554] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:09.843037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c20db-e486-49fc-93bc-0be35c95937e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:09.865558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:09.891294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a682-1c89-4f16-b92e-068f12f5ce35 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:09.932501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:09.933377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:09.933913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:09.934389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:09.960805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c20db-e486-49fc-93bc-0be35c95937e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.002220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.002852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.003128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.003399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.013777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.015235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.018429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.019868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-26533c3c-f06e-4c5a-8be8-6df20b25add0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:10.030403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:10.031366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:10.032255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:10.032255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:10.032255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:10.032255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 277/555] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:10.033148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.033446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.033835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.053442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:10.054654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:10.090790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.104884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.132748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 278/556] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:10.140673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.140940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 279/557] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 20 04:39:10.151384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.280725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a792-7941-4d6a-9378-5a6cb01dd5b2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.306059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e43885-1f4a-4caa-ac75-6e0dfb985753 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.339542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.340174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.340449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.340762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.353096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.354230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.354549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.354844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.355124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.355588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.356055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:10.356358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.357765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151724a-bb83-4171-8eaa-8de0d93c380c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.368834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 279/558] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:10.370149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4943b-bcd0-4742-93f5-f01a96776186 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.370436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4943b-bcd0-4742-93f5-f01a96776186 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.370708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4943b-bcd0-4742-93f5-f01a96776186 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.371891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 280/559] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:10.375846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd718e-388f-4d5c-84d8-eb3a267ca2ed None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.375846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd718e-388f-4d5c-84d8-eb3a267ca2ed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.375846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd718e-388f-4d5c-84d8-eb3a267ca2ed None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.375846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 281/560] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:10.376154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd718e-388f-4d5c-84d8-eb3a267ca2ed None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c1b56790-8908-48da-8996-c84af19d1839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:10.376503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:10.377428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:10.378325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:10.378325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:10.378325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:10.385615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 280/561] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:10.385615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11cd32-887c-4ffd-b365-45709b1e9d49 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.385615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11cd32-887c-4ffd-b365-45709b1e9d49 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.385615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11cd32-887c-4ffd-b365-45709b1e9d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.402674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ce11cd32-887c-4ffd-b365-45709b1e9d49 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:10.404312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ce11cd32-887c-4ffd-b365-45709b1e9d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:10.414678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.432786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.472924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 281/562] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:10.474219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.475086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.475358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.495058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd718e-388f-4d5c-84d8-eb3a267ca2ed None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.499923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:10.501156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:10.534306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.535270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.535748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.536263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.546242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.548449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:10.563211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:10.564151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:10.565137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 282/563] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:10.566458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.574650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 282/564] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:10.576011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.664006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.672852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7ed33-e7af-4b6e-997d-2d0923f86a89 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.674360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.695442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df5bc9-0bb3-412f-bb50-9e836d41328d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.712382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.712936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.713220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.713490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.714275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:10.714651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.716173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.728451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 283/565] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:10.729312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.736777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.738303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.738795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.739319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.753406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:10.758820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:10.759735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:10.760589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 283/566] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:10.761891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.842994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e377b41-e038-4062-9da9-557e3b38dcd3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.862999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be406c4c-7638-46d5-a7dc-c30a422b93b7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:10.882223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.882822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.883096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.883370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.884155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:10.884457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.885925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.898254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:10.898883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:10.899164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:10.899437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:10.901787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:10.903356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 284/567] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:10.904239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.907738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:10.909211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:10.920727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:10.921823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 284/568] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:10.922742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:10.930255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.008077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77f76b-d01f-4d78-b70e-c040cf627188 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.020000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbcc83-2ccc-4064-84aa-289c68945b71 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.041228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.041848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.042128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.042404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.053004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:11.054683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.055275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.055552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.055821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.056471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.056798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:11.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.058796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:11.059628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:11.060280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.060350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 285/569] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.060350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.069758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 285/570] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.070584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.123750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.143070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.163040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c44238-115d-4a33-9565-1bf1df1c0d3a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.169647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820457da-884d-4f7e-8f83-d1e071cfad50 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.197634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.198285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.198565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.198842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.203401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.204029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.204309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.204586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.208477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:11.213279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.214213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:11.215055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 286/571] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.215764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.215834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.216459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:11.232240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.233379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:11.234282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:11.234282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.234282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:11.234282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 286/572] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.234509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.314442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fce8d-6fd5-477c-83c3-76fd3c644946 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.337669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349812e-b59c-45f9-a603-50b693f00fe2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.346475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.347250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.347918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.348239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.348543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.349320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.349629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.351067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.359582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.364746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 287/573] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.366132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.371755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.372351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.372657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.372930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.373579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.373928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.375224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.383580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 287/574] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.384363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.473435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce77f3-a9bd-495d-9fb5-bb4c2513346a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.484782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3d0bf-5f70-4284-aacd-48edd05d1baf None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.506869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.507488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.507793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.508045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.517087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.517681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.517955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.518106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.518226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.518853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.519149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.519550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.520486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.528338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 288/575] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.529147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:11.534122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.535277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:11.536176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:11.536176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.536176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:11.536176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 288/576] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.536413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.563320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.587080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.633137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8d296-fe5c-4d81-bb06-49004c4660b1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.635304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be82f6-3a5c-417d-8f4f-f8c6fc636c84 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.666451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.667042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.667318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.667590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.668205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.668503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.668668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.669266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.669549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.669824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.669925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.670544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=7355daeb39aa4d8c8c1eb698fede46f4) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:11.670851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.672252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.678039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 289/577] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/role_assignments?user.id=7355daeb39aa4d8c8c1eb698fede46f4&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.678821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.693520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 289/578] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:11.694740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.799128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.807539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827671a-b29f-486c-bd7f-92fec82ee66d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.810113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ddbdb-6b77-40a0-abfb-091f7e9e02c7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.828781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:11.849276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.850336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.850834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.851177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.854062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:11.854700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:11.854983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:11.855629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:11.868093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.868093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.868340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:11.868661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:11.882428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.883468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:11.889478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:11.889478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.889478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:11.889478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 290/579] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.889478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:11.889781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:11.891228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:11.892152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:11.892152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:11.892152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:11.892152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 290/580] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:11.892152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:11.994495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b4f09-afbe-4b97-9723-c4f46366aa49 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:11.999338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22de1b6-934e-42d3-8fa9-3bf63128825b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.030673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.031301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.031581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.031857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.037613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.038611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.038872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.039293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.039708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.040059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.040718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.041163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.042697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:12.051662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:12.053171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:12.054088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 291/581] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:12.055652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.061012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 291/582] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.062320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.070863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.089412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.167848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06942bc2-f8ca-4b8b-9782-78b336d8bd05 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.187218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad352f5-587f-4e36-bfb9-659981547a43 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.202710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.203309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.203590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.203856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.204530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.204858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.206226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.217673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 292/583] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.218475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.231492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.232093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.232366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.232656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.233287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.233583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.234936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.247711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 292/584] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.248516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.313771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.315984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.330130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822178f4-fb9b-4afe-b66b-4ac6a4f0dfe8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.354848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463369-5120-47fe-8d36-72fa4f4ce277 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.368985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.369869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.370146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.370416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.371249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.371914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.373361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.390564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 293/585] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.390564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.397859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.398495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.399090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.399365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.407769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.409172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:12.420956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:12.421910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 293/586] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:12.422876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.496710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21496c7d-95aa-40d4-80bc-6b931faa306f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.527737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34deb12b-8152-498c-a880-94b9a6836349 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.531676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.532528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.533038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.533493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.541576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.543256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.553192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:12.555657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:12.556747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 294/587] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:12.558586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.570154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.570783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.571057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.571322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.576000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.581177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.582720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:12.594716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:12.595710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:12.596720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:12.600931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 294/588] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:12.601597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.671960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3f34e-d75a-4762-bbc6-de7ad07274ab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.706308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1c770-0ea5-49fc-84b8-02d6bd6cc2f4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.707375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.708376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.709559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.709895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.725696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=9809a4423cfb4ddda8d03e663f81b060, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.727148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.740647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 295/589] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:12 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:12.741340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.752295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.753887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.754412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.755516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.756874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.757432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.760095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.789670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 295/590] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.790841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.815223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.853728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:12.869971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e055ed-99e5-423e-bdb6-ef614aa62c10 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.910080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.910790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.911077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.911402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.912777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.913087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.914845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:12.937179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 296/591] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:12.938027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:12.940004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd0f76-eb30-4056-897f-b94c6a915f28 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:12.984458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:12.985465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:12.985981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:12.986452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:12.987375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:12.987879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:12.989518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.008888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 296/592] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:13.010193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.059230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b878137-ba8a-4dd9-9a1d-162a1adeeb99 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.081863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.105549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.106436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.106721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.106997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.121017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.122847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.127333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.131229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168bec32-908d-4a19-b8dd-39e02b45782d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:13.144219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.145237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:13.146175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:13.146175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.146175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:13.146175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 297/593] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.146416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.182685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.183741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.184032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.184300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.197972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.199372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:13.219841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.220895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:13.221851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:13.221851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.221851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:13.221851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 297/594] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.222656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.276177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd371e3-414f-47e1-b4bb-9db0715ab33d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.322209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.322835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.323114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.323383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.337905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.339843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.355751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177f38e-2dcb-401d-aed3-91fc7c65dac7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:13.359465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.360416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:13.361374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:13.361374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:13.361374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:13.361374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 298/595] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.362008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.398162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.411566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.416999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.417283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.417558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.418214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:13.418507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.419899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.437886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 298/596] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:13.438811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.454784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.493788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3db577-9124-45eb-997c-f3ed00c15dc9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.534069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.535509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.536012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.536490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.538094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:13.538606 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.540221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.548091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9873-bd20-4cc4-a43b-2643bdfd3f40 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.557846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 299/597] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:13.559096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.585120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.585734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.586347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.586347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.586993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:13.587298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.588738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.606305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 299/598] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:13.607349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.648119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.672940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0710f-923c-4f23-aa00-903ed31a452d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.691821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.710431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.711603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.712085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.712552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.719916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.721677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.730082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683715a-5877-4892-8101-f2c2028c3fb1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:13.733764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.734654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.735585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:13.737780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 300/599] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.738875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.786901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.787601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.787895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.788152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.804991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.806589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:13.822867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.823790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.824729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:13.829020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 300/600] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.830987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.862244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa9e81-d0ac-4b3b-a64e-17669247d3c5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.897951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.898961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.899438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.900185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.906812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.908238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:13.911220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:13.922177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:13.923107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:13.923996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 301/601] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:13.924480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:13.951949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f061e3f-dac1-43a3-9039-89d0e99d65f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:13.983365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:13.989634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:13.990778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:13.991058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:13.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:13.992179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:13.992528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:13.994060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deb29d-0126-4d5d-a0fb-40bcf4fc67e6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 301/602] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46c32-7446-43a3-89f1-20cf73e9fcfe None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46c32-7446-43a3-89f1-20cf73e9fcfe None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46c32-7446-43a3-89f1-20cf73e9fcfe None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 302/603] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:14.011695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46c32-7446-43a3-89f1-20cf73e9fcfe None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.029167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad689a-7528-4102-a2c9-82b225dcb5c3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.062637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.063239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.063520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.063790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.064471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:14.064796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.066225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.082319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 302/604] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:14.082319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.127344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46c32-7446-43a3-89f1-20cf73e9fcfe None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.156117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:14.165910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.166511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.166787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.167055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.167787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:14.168089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.169542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab874b-3078-4e9c-a6fe-7a1ac3135077 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.184770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 303/605] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:14.186115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.186470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.186751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.203536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e5704-673c-4989-9fe4-881f18d270bb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.205054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:14.206655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:14.239499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.240104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.240381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.240745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/ef0eb7d5e5b1416b8018e1a5c49a33e5/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.254823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=ef0eb7d5e5b1416b8018e1a5c49a33e5, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.256298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.257231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:14.276634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 303/606] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:14 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/ef0eb7d5e5b1416b8018e1a5c49a33e5/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:14.276634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.291594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 304/607] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:14.292768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.487876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd6799-e107-4185-979d-5b7c0be1d7db None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.514612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d69aa6-a324-4782-bda3-c8ff9ce5c12e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.532135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:14.534225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.534908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.535192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.535501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/f7cdae17359d4ba2ae149afbbd855ee7/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.557751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=f7cdae17359d4ba2ae149afbbd855ee7, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.559206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.562715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.564007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.564440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.564945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/2fe2203ea1f94623956a215190e4d79c/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.579573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 305/608] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:14 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/f7cdae17359d4ba2ae149afbbd855ee7/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:14.582686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.587227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=2fe2203ea1f94623956a215190e4d79c, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.588775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.615490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 304/609] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:14 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/2fe2203ea1f94623956a215190e4d79c/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:14.623549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:14.703631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f267c9b-5c41-460f-a419-30f31c073bea None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.743100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.743755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.744027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.744310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.757092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:14.767324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:14.768356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:14.769266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:14.772262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 306/610] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:14.775846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:14.901660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e97f7-dd26-4083-8380-a761c61027b5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:14.948310 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:14.949259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:14.949724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:14.950206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:14.951058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:14.951540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:14.953265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:14.962733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 305/611] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:14.968646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:15.067360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:15.078787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5013b-d3c8-4942-bba5-563d99c5128f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:15.137406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:15.138159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.138371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.138613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.150638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:15.152119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-7722b0ec-9d5f-4593-aa12-fb50910e5114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:15.180949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:15.181983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:15.183485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:15.183485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:15.183485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:15.183485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 307/612] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:15.187757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:15.219710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:15.335259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d853b7-daca-4390-bd22-3461b97e3a7c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:15.345942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0b2f-357a-4e94-956c-b9bc4bf4ba21 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.346215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0b2f-357a-4e94-956c-b9bc4bf4ba21 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.346493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe0b2f-357a-4e94-956c-b9bc4bf4ba21 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.348115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 308/613] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:15.353886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.354168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.354441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.374570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:15.375926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:15.393375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:15.393996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.394277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.394552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.395201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:15.395504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:15.397494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:15.423314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 306/614] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:15.426677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:15.494835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 309/615] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:39:15.549810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:15.569158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2f3f7-182c-47c1-a948-c0e6a51f9b11 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:15.617939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:15.618510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.618788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.619121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.628393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:15.629948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:15.651628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:15.652661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:15.653613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 307/616] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:15.660318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:15.745708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:15.773199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856a4b8-5c9b-43b1-999b-f8db809db1a2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:15.812646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:15.813307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:15.813595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:15.813864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:15.814597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:15.814898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:15.816327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:15.835536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 310/617] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:15.841098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:15.906324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.004236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395801c-9583-4fb9-9166-6b4adb60bc71 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.073752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.074389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.074669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.074943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.089132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1182f-3aee-4f05-bed8-d8d068a60451 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.101895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.103359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.112909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:16.132174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:16.133207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:16.134109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:16.134109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:16.134109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:16.135254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 308/618] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:16.142228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.153570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:16.162958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.163769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.164133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.164413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.186231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:16.195372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:16.196686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:16.197555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:16.198964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 311/619] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:16.209910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.350538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cbf6a-afc4-4d48-9a5f-7fdaf3e69652 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.402228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.402890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.403173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.403445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.404517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:16.404829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.406340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.413090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb74afe-5c36-481b-b6cc-46ac110787d3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.420630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 309/620] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:16.425623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.467038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.467650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.467922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.468190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.468875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:16.469171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.470520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.493789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 312/621] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:16.497992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.518269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:16.557027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:16.599695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ef62b-13f6-4917-97a4-c7825e6a7d37 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.636279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b8c2d-8b1a-4c74-aeab-d557d6b173ac None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.637737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.638716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.639092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.639475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.647377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.649014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:16.662639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:16.663644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:16.664563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 310/622] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:16.673128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.741860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.742605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.743000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.743347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.769666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.771947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.794293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5070810-feaa-4fff-a4de-eb88fc1b5685 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:16.809312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:16.810318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:16.811261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:16.811261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:16.811261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:16.811261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 313/623] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:16.817636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.829041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:16.867006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:16.867622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:16.867899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:16.868165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:16.868885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:16.869186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:16.870638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:16.896926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 311/624] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:16.904356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:16.979125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19031-53c6-44dd-b60a-e980de9640b8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:16.989455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.037160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:17.038353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.039506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.039963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.041098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:17.041590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:17.043321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74fd6f-6e3b-45bb-8bc9-416364a51e39 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:17.058087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 314/625] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:17.059444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6972c-9f7d-459c-a582-6aa17628460a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.059731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6972c-9f7d-459c-a582-6aa17628460a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.060004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6972c-9f7d-459c-a582-6aa17628460a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.064679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 315/626] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:17.065734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6972c-9f7d-459c-a582-6aa17628460a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:17.075831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96534295-cc7a-42d1-9800-3f54b6c691d4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:17.117537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:17.118196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.118479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.118751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/7355daeb39aa4d8c8c1eb698fede46f4/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.134593 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=7355daeb39aa4d8c8c1eb698fede46f4, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:17.136302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d12c0-1d93-4165-acaa-7c944378f9da None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:17.158819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 312/627] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:16 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/7355daeb39aa4d8c8c1eb698fede46f4/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:39:17.160396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4942-4d85-4543-a532-8ab7fb760e2d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.164663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4942-4d85-4543-a532-8ab7fb760e2d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.164663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a4942-4d85-4543-a532-8ab7fb760e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.207190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-dc8a4942-4d85-4543-a532-8ab7fb760e2d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:17.209343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dc8a4942-4d85-4543-a532-8ab7fb760e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:17.239013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6972c-9f7d-459c-a582-6aa17628460a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:17.258424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.317358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:17.318616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.318996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.319366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.335921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:17.337850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:17.348507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 313/628] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:17.350330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdcd4-be51-4c90-9f44-655ef4315959 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.350580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdcd4-be51-4c90-9f44-655ef4315959 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.350848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdcd4-be51-4c90-9f44-655ef4315959 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-607cc311-d1f2-4ae2-9b1f-a28a7d6d1dbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:17.353729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:17.354724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:17.355694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:17.356229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 314/629] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:17.356300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 316/630] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:17.360348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdcd4-be51-4c90-9f44-655ef4315959 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:17.360812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96ff8-9743-4758-be84-8f2b9128446f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.361096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96ff8-9743-4758-be84-8f2b9128446f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.361428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f96ff8-9743-4758-be84-8f2b9128446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.384641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-49f96ff8-9743-4758-be84-8f2b9128446f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:17.386426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-49f96ff8-9743-4758-be84-8f2b9128446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:17.415180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.478388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 317/631] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:17.479797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe29dc8-961b-4462-8a73-600bce1fd900 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.480156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe29dc8-961b-4462-8a73-600bce1fd900 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.480499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe29dc8-961b-4462-8a73-600bce1fd900 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.482071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 318/632] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:17.482701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdcd4-be51-4c90-9f44-655ef4315959 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:17.490971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.491221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.491445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.523901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:17.525374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:17.548659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:17.549288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.549549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.549824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.550558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:17.550864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:17.552868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:17.582875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 315/633] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:17.586589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:17.613998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.688078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 319/634] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:39:17.689117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:17.788303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948622d0-a4b6-4168-a311-f7e27bab0ec7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:17.847048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:17.848048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:17.848546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:17.849105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:17.912969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.919117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:17.921030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:17.936987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:17.951566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab3a99-0e92-481e-9a63-10c400d3c537 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.026889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 316/635] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:18.027826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.038915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.039480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.039755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.040025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.055779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:18.060675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.061663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:18.062524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 320/636] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.063256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.166713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14368a-6955-46df-b598-985ced53b851 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.180547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.205917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f7215-e1e4-4b1f-b062-3b390e4b0d36 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.207798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.209710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.210151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.210478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.224167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:18.229572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.230520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 317/637] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.231380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.253052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.254294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.254753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.255766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.268073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:18.273716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.275007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:18.276038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:18.278374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 321/638] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.278374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.288534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.338490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6ab48-3c8b-4806-92b3-eceaac37a4e9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.373150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.373923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.374201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.374472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.384053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82807e9b-fba0-4d2d-a3ff-3e17197bd11c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.385318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.386757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:18.402131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.403106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:18.404055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:18.404055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.404055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:18.404055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 318/639] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.404055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.430369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.431003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.431729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.432122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.432441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.433295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:18.433582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.435832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.450108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 322/640] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:18.450108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.517755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c099c-53cd-4c09-b445-bf0d25034788 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.519928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.552791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.553697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.554039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.554306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.554998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:18.555247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.556627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.557508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dce6ae-4a19-4806-985b-7cb930e447b5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.564799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 319/641] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:18.565598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.597995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.598826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.599719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.599998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.600858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:18.601166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.603304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.612517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 323/642] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:18.613322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.657764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.675282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981fc97d-5283-41e5-9f00-c3c234777715 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.710824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.711480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.711796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.712124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.712833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:18.713138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.714516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.716171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ef108-2f2e-47da-b48f-9ee7158afc76 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.731715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 320/643] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:18.731715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.749891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.757171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.757900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.758246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.758607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.769314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.771418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:18.785715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.787563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:18.788761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:18.788761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.788761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:18.791761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 324/644] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.792435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.843576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd353146-c594-46e4-941f-a4474ba44b81 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.881802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.882350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.882628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.882897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.895497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.897413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d41375-e4e6-4f74-9709-9c1ebf509b28 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:18.897553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.904726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:18.914008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.915175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:18.916427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:18.916427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.916427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:18.916667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 321/645] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.917587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:18.940646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:18.941540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:18.941820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:18.942099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:18.953965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:18.955399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:18.971407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:18.972397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:18.973804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:18.973804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:18.973804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:18.973804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 325/646] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:18.973804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.011142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.031807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab1aa9-60a1-4883-a496-59a803cb6e1b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.069090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.069715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.070033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.070254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.077325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.078749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.079589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedc044-361a-4b33-bada-c2be7da2824f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:19.089796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:19.090776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 322/647] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:19.091697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.122178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.122874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.123227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.123575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.124296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.124704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.126156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.139579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 326/648] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.140645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.166352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.212206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa30ef-c44d-4251-a9d1-f4fbd0c09bde None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.243893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.249039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.249642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.249939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.250200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.250824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.251125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.251966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b1bb-6b70-4181-9a86-96f0cf49ec47 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.252565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.264623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 323/649] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.269734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.288653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.289262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.289544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.289816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.290449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.290755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.292157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.310932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 327/650] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.310932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.384373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c906d7-c0ab-43af-aaec-6e91680a1ffc None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.403010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.421635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.422232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.422508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.422774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.423450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.423752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.425004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e9a3a-d5e0-4a66-a3c2-2c9f08647020 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.425217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.436441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 324/651] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.437274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.463635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.464253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.464527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.464827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.473130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.475183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:19.486294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:19.493198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 328/652] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:19.494252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.504573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.535592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9430c3-c04f-478b-8fbf-70118bc47c6a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.568567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.569205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.569478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.569751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.576812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.578239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.587757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cef7d8-9613-4c17-b22f-9beaa0f56380 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-7af1458c-9e9b-42b5-acaa-f2950e28e304 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:19.589383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:19.595551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 325/653] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf181-e2e1-4478-87ff-79b03d89af56 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.596844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf181-e2e1-4478-87ff-79b03d89af56 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.597963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf181-e2e1-4478-87ff-79b03d89af56 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.597963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 326/654] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:19.597963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cf181-e2e1-4478-87ff-79b03d89af56 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.623880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.624509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.624814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.625088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.632313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.633787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.634241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:19.644991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:19.646668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 329/655] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:19.647933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.698983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cf181-e2e1-4478-87ff-79b03d89af56 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.724249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.732643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.733242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.733518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.733783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.734492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.734795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.736200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.741928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 327/656] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.742695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.750556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99b2f9-5309-42d3-aa04-6b78061c5370 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.791956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.792572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.792881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.793154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.794007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.794436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.795933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e7f7-f544-4752-b586-f0985cf22139 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.821504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 330/657] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.823028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.823320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.823591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.843202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:19.844777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:19.852021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376e2c8-bc3d-4f9a-992e-2d6175295fb4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:19.854104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:19.892246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:19.892892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:19.893167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:19.893440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:19.894175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:19.894478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:19.895954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:19.916800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 328/658] 10.222.0.27 () {56 vars in 1137 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:19.916800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.924848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 331/659] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:19.925594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:19.998655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.041827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d481-35aa-45fd-94b8-4f0755a9c892 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.052910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a8a56-80b5-49ee-93c0-9ca8abf9ca1e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.090766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.091514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.091906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.092227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.093021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:20.093507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.095024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.095982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.096839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.097090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.097467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.117980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 329/660] 10.222.0.27 () {56 vars in 1139 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:20.118841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.150086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.182032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.183619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.246446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1cae1-e3d1-4918-b22e-949d3405a618 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.251297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.281096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.281695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.281970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.282240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.304011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 332/661] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:20.304011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.349339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.350883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.368947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.416830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6d63-e00c-4745-91da-aac67859d750 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.451805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.452408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.452710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.452980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.463139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 330/662] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:20.463139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.490538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.539199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.540748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.569999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42079-6316-4e07-87b6-8f639cd40aab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.588325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.606927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.607555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.607836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.608107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.621505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:20.626626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:20.627805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:20.628953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 331/663] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:20.629912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.649890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 333/664] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:20.650944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.718693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.732039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7061c-f691-4acb-a17f-6459f7c098fd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.761871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8456f-230f-4753-8652-2fc6dd6ab6bc None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.769852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.770437 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.770748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.771554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.783023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.784517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.800840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.801390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:20.801498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:20.802955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:20.803849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:20.803849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:20.803849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:20.804031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.804031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.804157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 332/665] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:20.804157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.817827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.819286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:20.834581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:20.835740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:20.836698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:20.836698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:20.836698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:20.836698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 334/666] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:20.836947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.842510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.931022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552d172-5e49-4c11-b842-4ec1b7ce34ec None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.949063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf84-cf1c-43ab-8358-bbdc7aa05e90 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:20.962822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:20.967842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.968456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.968771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.969050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:20.969712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:20.970021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:20.971456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:20.980281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 333/667] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:20.981061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:20.988700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:20.989328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:20.989636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:20.989910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.001077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.002502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:21.017266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:21.018234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:21.019160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:21.019160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:21.019160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:21.019160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 335/668] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:21.019160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.081701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.092405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6346df-477b-4df5-a325-640243ad095c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.125275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42489d-219e-471c-9656-ad002b136ffb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.132422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.133065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.133342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.133609 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.134250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.134549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.135919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.148263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 334/669] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.149114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.169843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.170450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.170744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.170992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.171620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.171950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.173311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.190000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 336/670] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.190000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.220976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.267815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68975e28-c789-4669-873b-5f56ea954b5a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.298665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d796f23-6674-40ec-8b5a-a8eac53a9978 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.301010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.301631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.301936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.302217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.312130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.313589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:21.328281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:21.329279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:21.330191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:21.330191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:21.330191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:21.330191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 335/671] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:21.330191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.334420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.341780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.342389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.342687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.342965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.343608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.343909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.345302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.358543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 337/672] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.359358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.438706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840cdf0-5b5a-4b97-a0c1-c741c817989f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.450070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.467127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368a820-70ab-4ccb-8648-cb52fe6f0fe1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.473736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.475118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.475579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.476057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.482872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.484332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:21.496587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:21.497545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 336/673] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:21.498517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.506854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.507489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.507767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.508032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.514838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.516447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:21.530210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:21.531103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 338/674] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:21.532028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.569682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.612881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58985f0-77e5-4bdb-9683-e8290db4460e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.639041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdb30-ff09-48a8-8d5b-c38cd0cee2e3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.647634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.648239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.648825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.648825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.649441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.649992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.651351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.663849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 337/675] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.664824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.684424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.685080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.685356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.685624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.692838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.694269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:21.706613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:21.707566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 339/676] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:21.709926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.720539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.772132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac162e-e794-4a0f-9940-604e045b083a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.798156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.809868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.810475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.810751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.811018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.811814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.812115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.813690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.815559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c77dc0-ab6c-4aad-85be-dd610648a707 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.830068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 338/677] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.830068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.857128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.857850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.858190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.858532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.859285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:21.859591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.861056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.873209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 340/678] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:21.873945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:21.936480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fce3f5-98e3-4f0f-a90f-df2c2c47b41b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.948783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:21.974894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:21.975546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:21.975827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:21.976101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] PATH_INFO: `/v3/users/swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:21.984378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:21.985907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:21.989040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1af3d-426e-42fb-9c4a-ee10d692afb8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:21.998323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:21.999551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:22.000763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:22.007682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 339/679] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:22.007682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.031439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.032179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.032583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.032976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.033756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:22.034159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.035684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.047786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 341/680] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:22.048690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.057597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:22.114097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b5cc-c10c-480a-924f-77705329465a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.147950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.148474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.148778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.149053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.150049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=ef0eb7d5e5b1416b8018e1a5c49a33e5) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:22.150353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.151811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.157028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6792d-7e5a-4c28-afcf-b38e4826eadb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.157903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 340/681] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/role_assignments?user.id=ef0eb7d5e5b1416b8018e1a5c49a33e5&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:22.158677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.192080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:22.203104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.203720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.204005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.204279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.205117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=f7cdae17359d4ba2ae149afbbd855ee7) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:22.205432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.206917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.213738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 342/682] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/role_assignments?user.id=f7cdae17359d4ba2ae149afbbd855ee7&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:22.214593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.298301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2862a-a950-4e28-ac93-f0d68a757fb9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.310373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:22.343138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.343897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.344234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.345138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.346282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:22.346668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.348229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a393912-260f-482c-b9e4-c55aec892056 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.348712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26623b6-64ec-418d-8458-f87439fbc79d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.374302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 341/683] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:22.376643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c07a3-038a-4a65-954e-e7f1e770ed9c None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.376643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c07a3-038a-4a65-954e-e7f1e770ed9c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.376643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c07a3-038a-4a65-954e-e7f1e770ed9c None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.378373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 342/684] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:22.381932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c07a3-038a-4a65-954e-e7f1e770ed9c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.394041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.394668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.395399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.395709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.396843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=2fe2203ea1f94623956a215190e4d79c) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:22.397165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.398891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e522d-211a-4b90-aded-fca16ac6e2aa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.413811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 343/685] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/role_assignments?user.id=2fe2203ea1f94623956a215190e4d79c&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:22.415006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.415287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.415560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.437112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:22.438430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:22.514634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:22.519844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c07a3-038a-4a65-954e-e7f1e770ed9c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.543539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 344/686] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:22.566078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.566699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.566974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.567243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.610326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:22.649554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.651825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.761850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 343/687] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:22.767193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.854885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:22.902852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c42c0-efdb-4606-950b-87d4e7cc2dab None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:22.957745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:22.960259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:22.960795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:22.961265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:22.983034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:22.983499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:22.992737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:23.024588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:23.025886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:23.029154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:23.029154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:23.029154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:23.029154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 345/688] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:23.031583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.049157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236fad-7007-407a-8a05-1ab7e1779218 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.117006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.119490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.120223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.120547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.123645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.153367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:23.163047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:23.164023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:23.165242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 344/689] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:23.173631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.189202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f434fe-d21f-4d44-a097-a2c7f4a8a05a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.237785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.239893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.239893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.239893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.239893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:23.239893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:23.241297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:23.267703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 346/690] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:23.273609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.387971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874300ce-2011-4d2a-8dbb-21c40150ff8b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.425113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.433265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.434185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.434482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.435088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.436368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:23.436730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:23.438131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:23.450348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 345/691] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:23.455837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.496047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.504149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669afb2-bd04-47e9-9c27-ad6341fff0be None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.571940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.572569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.572864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.573129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.587031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:23.588485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:23.603581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:23.604632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:23.606123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 347/692] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:23.610526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.640194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683eed0-2dcf-4ab8-a2f5-fc348dbbb9f7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.706617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.707164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.707438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.707702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.723839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:23.725317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:23.752447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:23.753478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:23.754369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:23.754369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:23.754369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:23.757946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 346/693] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:23.760614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.774279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd933b5-cc28-484a-8f7d-2f0b32d50e1a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:23.810901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:23.811506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:23.811784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:23.812054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:23.816622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:23.816622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:23.816622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:23.827967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 348/694] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:23.834514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:23.856775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.861943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:23.965406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede7df-635c-443b-9b72-4072bc8341a8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.007526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:24.008140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.008249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb85816-077a-4a4a-81f2-2aa695210bcc None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.008435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.008713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.009357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:24.009659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:24.011033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:24.023461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 347/695] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:24.033601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:24.070838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:24.071447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.071732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.072004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.072748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=7355daeb39aa4d8c8c1eb698fede46f4) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:24.073058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:24.076415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:24.088259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 349/696] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:23 2022] GET /identity/v3/role_assignments?user.id=7355daeb39aa4d8c8c1eb698fede46f4&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:24.164902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:24.173015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f3602-db86-423f-abed-9542c90c0a98 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.225959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:24.226510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.226793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.227074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.237279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:24.238732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:24.251090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:24.252136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:24.253761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 348/697] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:24.258962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:24.370762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:24.390479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05312cd4-631f-4463-853e-eb7dad789950 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.438249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:24.438869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.439154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.439425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.440157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:24.440460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:24.441954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:24.460665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 350/698] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:24.467163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:24.587649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e2db-5443-4663-ad16-4faa20a6eb8b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.622968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:24.623579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.623859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.624129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.654312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:24.698291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:24.699762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a4a4-a0df-4eee-b72d-0702452733e7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:24.827485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 349/699] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:24.832716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:24.931460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac014b5-6007-4083-b801-a16b13edbd8f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.931976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac014b5-6007-4083-b801-a16b13edbd8f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.931976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac014b5-6007-4083-b801-a16b13edbd8f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.933551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 350/700] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:24.946766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64362be-a521-4219-a619-0cfac86661d5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:24.947063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64362be-a521-4219-a619-0cfac86661d5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:24.947318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64362be-a521-4219-a619-0cfac86661d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:24.968877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef4c5-01cd-461d-822e-decf9fd7d756 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:24.981914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b64362be-a521-4219-a619-0cfac86661d5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:24.983110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b64362be-a521-4219-a619-0cfac86661d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:25.011767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.012565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.013032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.013304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.021893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.027147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.028624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d2d77da5-aaec-4e46-8243-55cac0fda2bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:25.049222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:25.050236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:25.051122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:25.051122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:25.051122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:25.055449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 351/701] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:25.056788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d66cd-3ec8-429a-9b15-39851a680a3d None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.057068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d66cd-3ec8-429a-9b15-39851a680a3d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.057342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d66cd-3ec8-429a-9b15-39851a680a3d None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.058867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 352/702] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:25.059574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d66cd-3ec8-429a-9b15-39851a680a3d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.065271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.080313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 351/703] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:25.081697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.081986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.082261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.102555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:25.103746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:25.187264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 352/704] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:25.266276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d66cd-3ec8-429a-9b15-39851a680a3d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.310594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.311197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.311475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.311746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.312382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:25.312776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.314147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80587-5197-446d-853f-05cb59edea7b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.332190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 353/705] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:25.338259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.344069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0a1a-01ad-4990-9113-1eb95e552725 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.344357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0a1a-01ad-4990-9113-1eb95e552725 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.344659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd0a1a-01ad-4990-9113-1eb95e552725 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.346255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 354/706] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:25.353391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.353897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.354184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.374453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:25.375801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:25.469032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.503059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 355/707] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:25.503919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.509759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf106ad2-6736-4b1a-9a7c-188753d90aab None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.543432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.559028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.559880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.560394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.560860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.568060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.569813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:25.581346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:25.582271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:25.583733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:25.586628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 353/708] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:25.587754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.634629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f348a3-38bd-4ab1-aa8e-084855c6b932 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.672358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.673359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.673894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.674365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.685474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.687594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:25.703600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:25.704588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:25.705525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:25.705525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:25.705525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:25.706119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e03ddf-128d-42e5-9c21-124c6204f672 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.707823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 356/709] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:25.708919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.745515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.750077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.750793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.752115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.752395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.753008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.754369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:25.766244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:25.767345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:25.768193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:25.772140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 354/710] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:25.773008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.821380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.825755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cf947-40b3-4828-93c5-b713b2778b58 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.860301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.861346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.861346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.861534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.862116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:25.862422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.863843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.884485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 357/711] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:25.884691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446b4be-fbb4-4954-bec2-26350f972139 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:25.884777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.918592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:25.919197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:25.919468 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:25.919737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:25.920370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:25.920707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:25.922064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:25.933878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 355/712] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:25.934670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:25.984044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:25.994879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3833f-f2aa-4337-81be-df30ff8abde4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.035437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.035970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.036281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.036553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.037208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:26.037510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.038880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.043193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 358/713] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:26.043964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:26.044704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18269b6e-3b49-4a44-9d35-62d17352f56f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.049552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:26.084981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.085605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.085909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.086179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] PATH_INFO: `/v3/services/network` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.086816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.088173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:26.097633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:26.098701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:26.099573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 356/714] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:25 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:26.100365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:26.160483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8d3d-ff5d-479f-a37e-391dded36e26 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.199516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.200116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.200939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.201201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.202182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:26.202490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.204155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.216767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 359/715] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:26 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:26.216767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:26.217054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffef191-8f8c-400d-9744-75cb87b2d050 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.250420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.251037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.251316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.251584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.251697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:26.252359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.253872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.284633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 357/716] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:39:26 2022] POST /identity/v3/users => generated 254 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:26.284633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:26.345107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:26.426449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e59db-3034-4b56-bb09-09633d1fa841 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.487574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.488462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.488785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.489056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.490253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:26.490555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.490658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956c5dc-509a-4662-8acd-8bdd52b06563 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.491935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.498197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 360/717] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:26.537244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.537870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.538148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.538419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.539067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] RBAC: Adding query filter params (name=network) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:26.539374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.540904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d2991-7baf-42e6-9ff3-cc982d893c63 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.547079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 358/718] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:26.553217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:26.651509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:26.672246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9f69f-0ce7-42b1-bd44-e1df5c82c274 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:26.713096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:26.714101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.714577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.715043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.715891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] RBAC: Adding query filter params (type=network) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:26.716424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:26.718021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d014-a023-4d61-89cd-f8b3c6c67a3b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:26.724632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 361/719] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:26.873990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba217-e7a7-41dd-8376-b812dde47607 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.874272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba217-e7a7-41dd-8376-b812dde47607 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.874553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ba217-e7a7-41dd-8376-b812dde47607 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.876171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 359/720] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:26.882143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:26.882683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:26.882683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:26.910247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:26.912005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:26.995251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 362/721] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:27.273470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba217-e7a7-41dd-8376-b812dde47607 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:27.400514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:27.422219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ba217-e7a7-41dd-8376-b812dde47607 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:27.476120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:27.476781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:27.477062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:27.477333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:27.491631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:27.496402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:27.497423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:27.498280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 360/722] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:27.503556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:27.612669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:27.646822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cc701-c08f-4345-8976-6b37d7adf6af None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:27.692082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:27.692755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:27.693102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:27.693380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:27.694235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:27.694542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:27.695930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:27.709509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 363/723] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:27.715232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:27.862235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317348c9-dc60-41f9-8e77-d4dae34e5acd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:27.909555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:27.910097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:27.910372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:27.910646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:27.924093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:27.925563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:27.938141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:27.945724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:27.946707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:27.947693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:27.947693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:27.947693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:27.947693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 361/724] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:27.952897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:28.107439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863e410-736b-4d0b-8328-d1d2e1c32e9e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:28.161974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:28.162545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:28.162825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:28.163102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:28.163746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:28.164050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:28.165445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:28.182434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:28.185762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 364/725] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:28.191068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:28.323776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6acc-167b-4b85-b47d-2680e3148cb4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:28.372448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:28.373108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:28.373385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:28.373654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:28.383881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:28.385338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:28.400458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:28.401475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:28.402461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 362/726] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:28.407341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:28.542477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c645e5-c508-4461-bc52-6e6b2af41aaa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:28.577916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:28.578530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:28.578811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:28.579082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:28.579811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:28.580118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:28.581582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b7e24-e8f4-4056-ae24-9e2ab58e71a9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:28.601004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 365/727] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:28.606411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:28.701923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:28.758882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890f3f6-220f-46f7-bfaf-2a8616bea2f6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:28.810312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:28.810855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:28.811139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:28.811464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:28.906784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:28.908319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:28.987324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.021061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f43a9-09bd-452a-9878-b44d9f206f6b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.022006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f43a9-09bd-452a-9878-b44d9f206f6b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.022428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f43a9-09bd-452a-9878-b44d9f206f6b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.024202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 366/728] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:29.030302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28722-29dc-4101-99ec-49805412c501 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.030781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28722-29dc-4101-99ec-49805412c501 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.031222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28722-29dc-4101-99ec-49805412c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.041415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 363/729] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 20 04:39:29.046807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.053641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-48f28722-29dc-4101-99ec-49805412c501 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:29.055090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-48f28722-29dc-4101-99ec-49805412c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:29.122937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.144889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 367/730] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:39:29.202376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ba95-0238-4e53-a520-b677a437a7ee None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:29.241558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:29.242196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.242480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.242753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.256644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:29.258263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-9d3566f5-ebac-4102-b8e7-07886e2db78a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:29.276562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:29.277673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:29.278636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:29.278636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:29.278636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:29.278636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 364/731] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:29.283756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f28722-29dc-4101-99ec-49805412c501 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.316672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296a5f-c48d-4cbe-bc09-7fea52908c88 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.317114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296a5f-c48d-4cbe-bc09-7fea52908c88 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.317524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296a5f-c48d-4cbe-bc09-7fea52908c88 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.319269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 365/732] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:29.325156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f193-ae9e-4478-9ff9-dfbfb665bf08 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.325738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f193-ae9e-4478-9ff9-dfbfb665bf08 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.325914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933f193-ae9e-4478-9ff9-dfbfb665bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.347998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6933f193-ae9e-4478-9ff9-dfbfb665bf08 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:29.350391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6933f193-ae9e-4478-9ff9-dfbfb665bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:29.359236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.437431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 366/733] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 20 04:39:29.439079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d902-6244-42da-a036-f1687edb49d5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.440021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d902-6244-42da-a036-f1687edb49d5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.440795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5d902-6244-42da-a036-f1687edb49d5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.443530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 367/734] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:29.449902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.450432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.450930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.464990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f28722-29dc-4101-99ec-49805412c501 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:29.474242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:29.475870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:29.556829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 368/735] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:29.579857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:29.580483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.580785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.581229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.582099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:29.582549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:29.585007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783d436-d195-4165-90c3-ae2a0b99be0e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:29.585729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.608742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 368/736] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:29.610296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd2918-690c-49ad-a429-bfde0760f6f4 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.610760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd2918-690c-49ad-a429-bfde0760f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.611180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd2918-690c-49ad-a429-bfde0760f6f4 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.613506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 369/737] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:29.615097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd2918-690c-49ad-a429-bfde0760f6f4 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.670359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.682229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.720358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f83dbd-29b2-41ed-9316-31cb81e1bac7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:29.731647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd2918-690c-49ad-a429-bfde0760f6f4 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:29.753826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:29.754449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.755293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.755523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.768369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:29.769252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:29.769872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.770161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.770514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-f617385a-e9b3-47cc-9442-bc9969095a45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:29.775002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:29.776030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:29.776911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 369/738] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:29.778107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.778217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:29.778292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.778362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.779465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:29.791624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:29.793327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:29.795545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:29.797299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 370/739] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:29.797897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:29.798093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.798845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:29.872229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 370/740] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 20 04:39:29.873463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.913707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a785753-675f-4e59-8d28-fac633c59962 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:29.925806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.947615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:29.954172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:29.954784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:29.955063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:29.955333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:29.955981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:29.957375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:29.978126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 371/741] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:29 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:29.979419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:29.998968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71361acd-8977-4f4d-8643-e8b711000be6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.051060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.051973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.052346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.053095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.053919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:30.054551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.056189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.068516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 371/742] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:30.069800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.138339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef80c-e6ed-4127-98b9-df2fd082934f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.185198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.185813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.186091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.186357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.187036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:30.187339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.188852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.203762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 372/743] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:30.204582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.206431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7ca6-00bb-428d-a4a4-5f1cfd81396a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.228957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.236172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.251818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.252465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.252781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.253056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.266818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:30.275594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:30.276558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:30.277441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:30.278104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 372/744] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:30.278785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.328688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb699674-3b17-4568-8bbe-940cd539fa3a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.365197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.366314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.366772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.367317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.378072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.379512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.383998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f04f13-f5f8-4bf1-8a84-39dc237d9e5f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:30.395075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:30.396801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:30.398125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:30.398125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:30.398125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:30.398125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 373/745] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:30.398125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.424366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.425102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.425384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.425662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.426311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.428270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.456631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 373/746] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:30 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:30.456631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.507834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.519960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.540570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab1a5-bc3b-4e57-9c64-ce25968248ed None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.602176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.602845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.603183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.603520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.604419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=7355daeb39aa4d8c8c1eb698fede46f4) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:30.604771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.606363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.609423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7031d87-6758-4c92-acc6-91f074af1bd7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.613237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 374/747] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/role_assignments?user.id=7355daeb39aa4d8c8c1eb698fede46f4&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:30.614777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.649478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.650840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.651321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.651847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.652804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:30.653346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.655278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.665125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 374/748] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:30.671673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.739807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e3e39-b96c-4b7e-bfaf-103574ed3dfa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.781448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.782514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.782998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.783514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.784437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:30.785054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.787342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.797164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b03f2b-8abe-4fc1-9c9d-1839990bd211 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.797918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.800191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:30.803596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 375/749] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:30.812204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.849414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.850417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.850914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.851418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.864202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.866266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:30.883060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:30.884074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:30.885022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:30.885022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:30.885022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:30.887664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 375/750] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:30.894736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:30.933792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab51b5-b2d6-4ac4-aec7-f3082eb4ee3a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:30.975882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:30.976426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:30.976828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:30.977254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:30.990280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:30.991724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:31.011068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:31.012070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:31.013059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:31.015195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 376/751] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:31.020690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.061269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54563b34-fa70-4def-aab8-e3fa4879e4e5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.125874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.126494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.126763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.127029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.127671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:31.127966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.128940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:31.129351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.165427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481c1f9-792b-4006-bf71-937328c5601a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.203102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.203902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.204283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.204689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.205561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:31.205971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.207647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.230449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 377/752] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:31.238790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.308749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 376/753] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:31.326816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.353664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddacd4-a7c0-4701-907a-70b651f1b668 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.370665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:31.391564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:31.430684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.431330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.431612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.432030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.453972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.455454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:31.494840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:31.495894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:31.496849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:31.496849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:31.496849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:31.496849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 378/754] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:31.501451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.518283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32aeded-731e-47d1-b5b9-86959e59b65e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.559955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.560587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.561700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.562804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.576016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.577555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:31.607054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:31.608152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:31.609151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:31.612989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 377/755] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:31.615648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.704286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab3476-a58f-4972-8c81-4b910727e5b8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.740261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.741005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.741289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.741564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.742203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:31.742507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.744015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.761604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:31.762296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 379/756] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:31.763732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.801106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:31.856329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f699a-b7c8-4ef9-ac8e-18b6394490cd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.890097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42154b-b35f-498b-badf-7d665d43f4a3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:31.903915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.904527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.904826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.905087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.905820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:31.906110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.907542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.938247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 378/757] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:31.944657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:31.957119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:31.957799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:31.958097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:31.958370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:31.970312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:31.971765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:31.998452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:31.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:32.000352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 380/758] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:32.006507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:32.112423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:32.118300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab34753-82e8-49fa-85ef-b1d77ab87e51 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:32.120992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb4b3-0db5-4307-9e46-1bdc4f43a073 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:32.157116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:32.157874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.158094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.158371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.159063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:32.159372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:32.161059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:32.179064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 381/759] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:32.180869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:32.184565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:32.213306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:32.213929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.214211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.214482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.291302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7154969-f1fc-494e-9ba2-e7196d46517a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:32.330255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:32.330925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.331204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.331474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.334119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=9809a4423cfb4ddda8d03e663f81b060, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:32.335633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f0185-e572-4315-9189-29e06ec294fa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:32.358838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 382/760] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:32 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:39:32.420706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:32.422116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:32.498181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34582f-49bc-45ac-b68a-851acd20bc88 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.498508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34582f-49bc-45ac-b68a-851acd20bc88 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.498760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34582f-49bc-45ac-b68a-851acd20bc88 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.502235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 383/761] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:32.510885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.511168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.511441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.549149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:32.555662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:32.564696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:32.592682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 379/762] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:39:32.597141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:32.651021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:32.678364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 384/763] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:32.732553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffacd6-ced8-487e-9610-967a25643ec9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:32.772271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:32.773287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.773771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.774239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.789709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:32.791515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:32.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:32.811480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:32.812395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:32.812395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:32.812395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:32.814464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 380/764] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:32.820446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:32.934930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b192f-593c-4247-a920-3a750e794e80 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:32.980052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:32.980686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:32.980973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:32.981247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:32.981882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:32.982179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:32.983549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:32.992713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:32.999687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 385/765] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:33.006202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.052281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.175207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.195481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc747c4-0746-494c-b435-826a79afeaba None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.222703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b75ad-9d95-4b6c-ad0f-9f8c5f261d0b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.234166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.235136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.235602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.236088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.243887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.245699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-a6cc794e-9303-49bd-9ce8-47cc404abaa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:33.257996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:33.258934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:33.259976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:33.262014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 381/766] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:33.263124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.263679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e020b-0dff-4303-80ce-10affc7862d8 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.263839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.264215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.264473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e020b-0dff-4303-80ce-10affc7862d8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.264575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.265076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e020b-0dff-4303-80ce-10affc7862d8 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.265371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.266820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.271291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 382/767] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:33.272307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e020b-0dff-4303-80ce-10affc7862d8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-51597f85-7ed2-47c3-87cf-90a11bbbcbc8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:33.280988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:33.282013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:33.282892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 386/768] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:33.289423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.289943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.290400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.310917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:33.312436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:33.376516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.396811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 387/769] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:33.396811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.412185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e020b-0dff-4303-80ce-10affc7862d8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.461888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.463017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.463301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.463579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.464549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:33.464886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.467467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab1303-7774-4bae-aca7-8d627d04515d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.482516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 383/770] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:33.483738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de7d72-37c9-4cc0-a2c5-3e9df752c5b6 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.484022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de7d72-37c9-4cc0-a2c5-3e9df752c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.484294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de7d72-37c9-4cc0-a2c5-3e9df752c5b6 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.488730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 384/771] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:33.490607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de7d72-37c9-4cc0-a2c5-3e9df752c5b6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.497143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.508156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073abe8b-3743-4a32-8b7b-408bd17ff298 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.548893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.549904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.550251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.550633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.551332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:33.552323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.553991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2e10-6090-40ff-aa2a-a9f486499ebf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.563196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 388/772] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:33.563196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.563196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.563196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.582250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:33.583974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:33.627467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de7d72-37c9-4cc0-a2c5-3e9df752c5b6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.643026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.660626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 389/773] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:33.662072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.682080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.685191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.685682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.686150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.687084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=aab2ee7790444760b8333226be499621) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:33.687594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.689280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.706120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 385/774] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/role_assignments?user.id=aab2ee7790444760b8333226be499621&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:33.707224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.773527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df1449-74d6-4192-9fad-1b1236a969f3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.814045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.814668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.814948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.815219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.815976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:33.816282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:33.820628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:33.824468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 390/775] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:33.830360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:33.849879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.891972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6ed76-1061-407e-ac94-2352d6c38c4e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:33.944434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:33.946093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:33.948368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:33.948945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:33.964504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:33.975795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:33.981600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:33.982721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:33.983790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:33.986783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 386/776] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:33.987840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.039117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94aa81-9c2d-4e2e-8194-1688bbe5e4f9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.077987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.078672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.078952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.079226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.079921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] RBAC: Adding query filter params (interface=public, service_id=553c83a282d24fd0bd2f08966dcbc6f3, region_id=RegionOne) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.080227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.081752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.092214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 391/777] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/endpoints?service_id=553c83a282d24fd0bd2f08966dcbc6f3&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:34.093420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.119552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951cca-5a84-4306-9d7c-b03bae779505 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.148257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.163428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40562-cd22-49de-a874-463198213630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.164148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40562-cd22-49de-a874-463198213630 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.164431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40562-cd22-49de-a874-463198213630 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.164749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea40562-cd22-49de-a874-463198213630 None admin] PATH_INFO: `/v3/services/network` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.165604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea40562-cd22-49de-a874-463198213630 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.166958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea40562-cd22-49de-a874-463198213630 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-2ea40562-cd22-49de-a874-463198213630 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:34.177448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:34.178490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:34.179352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:34.183457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 387/778] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:33 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:34.184449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40562-cd22-49de-a874-463198213630 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.227874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d24b2-a037-4758-a20f-91927203b68a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.247744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.271264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.272082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.272505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.272901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.273685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.274581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.276839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.292834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 392/779] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.293886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.309034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea40562-cd22-49de-a874-463198213630 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.353115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.353979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.354571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.354998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.356149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.357997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.364931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 388/780] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.365917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.421517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb567226-fa68-4bd9-bf70-d6d7666ecc48 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.442822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.484662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.487659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.488159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.488588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.489984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] RBAC: Adding query filter params (name=network) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.490735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.493700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.506700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 393/781] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.513170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.536662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03461fa5-8428-4c92-add3-48b712043cf0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.572718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.572718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.572718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.572718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.589837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.593753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.612094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:34.623860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:34.624938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:34.625858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:34.625858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:34.625858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:34.625858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 389/782] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:34.634160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.695950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb15a2-3908-4dcf-8e8d-79521b89e291 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.741753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.742861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.743457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.743957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.744811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] RBAC: Adding query filter params (type=network) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.745320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.746907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.753164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 394/783] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.756322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc9d19-0b82-47a6-8fe7-3af03f67bb70 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.760538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.773839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.820732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.821772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.822284 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.822815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.823704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.824227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.826015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.852188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 390/784] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.857877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:34.893434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc32e26-6a51-4c4b-bb35-b36095022b6b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:34.901100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:34.937543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:34.938548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.939115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.939589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.940484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] RBAC: Adding query filter params (interface=public, service_id=d64af1cbf85c4e929db246d5bf220b2e, region_id=RegionOne) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:34.940892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:34.942447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f61976-cdea-4cda-b81b-175363455ae6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:34.948338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 395/785] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/endpoints?service_id=d64af1cbf85c4e929db246d5bf220b2e&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:34.953488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a00b-9508-4f91-ab6f-b0e68e110400 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:34.953852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a00b-9508-4f91-ab6f-b0e68e110400 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:34.954198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a00b-9508-4f91-ab6f-b0e68e110400 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:34.955803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 396/786] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:34.960176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a00b-9508-4f91-ab6f-b0e68e110400 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.014782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc5a7d-1258-441f-8d91-e6cf46c00fba None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.057121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.058154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.058666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.059163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.068125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:35.069966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-88be8b9e-528f-43b8-8fe5-a0ab867823c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:35.083779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:35.084798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:35.085785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:35.086339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 391/787] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:35.087565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.087676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.087894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.106583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:35.107829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:35.132821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:35.185620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 392/788] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:35.186667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.237825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a00b-9508-4f91-ab6f-b0e68e110400 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.307442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.308997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.309462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.312357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.313191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:35.314699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:35.325200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 397/789] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:34 2022] GET /identity/v3/services => generated 772 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:35.372811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ddc9b-51b7-40bd-aa01-bae1a94c179d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.428169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.429128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.429597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.430014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.430905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:35.431358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:35.433017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:35.458711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 393/790] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:35.464701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.505888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:35.516633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.544662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:35.597018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d5add-d316-439f-9d53-8c7cd6a23536 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.643250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.643473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.643753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.644034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.662300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:35.664116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:35.667769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470cc62-58d8-41b1-9ad1-30b7fcf9d02a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:35.682738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:35.683797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:35.684769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:35.684769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:35.684769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:35.684769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 398/791] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:35.689910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.714579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.715219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.715502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.715773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.729754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:35.734971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:35.735924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:35.737203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:35.742353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 394/792] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:35.752344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:35.843854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:35.870510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fc868-93c5-49a9-af39-4f6f98391560 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.904386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:35.945636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:35.946651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:35.947909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:35.949786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:35.950647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:35.951167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:35.953346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:35.959624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f0072-694f-4ede-a763-437176e19ccb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:35.979976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 399/793] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:35.987020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.016496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.017537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.018054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.019799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.020745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:36.022169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.026745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c300e7e-1102-4bbb-ad66-87aa476f6620 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.043388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 395/794] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:36.045281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5067-d49a-4997-acdb-88d8b781db24 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.045571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5067-d49a-4997-acdb-88d8b781db24 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.045845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5067-d49a-4997-acdb-88d8b781db24 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.052193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 396/795] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:36.052339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf5067-d49a-4997-acdb-88d8b781db24 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.119915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdde562-ac04-4d65-ba14-3dbcc39795ae None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.164675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.165622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.166126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.166602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.169165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf5067-d49a-4997-acdb-88d8b781db24 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.174162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.175870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2c5a5403-ff06-4da5-821d-bb5ffedbdb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:36.187604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:36.188532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:36.189497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:36.191870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 400/796] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:36.193848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.194338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.194811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.206042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.206676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.206958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.207232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.212952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.215394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.216384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:36.217739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:36.220358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.222056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:36.238863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:36.240007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:36.241189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:36.241189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:36.241189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:36.247278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 397/797] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:36.248247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.292055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 401/798] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:36.293146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.356746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090deeed-f767-4ea8-9d96-7f72141505ec None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.396039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2e0c7-ec1a-412f-b78f-469b923c69b6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.405401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.406297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.406746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.407155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.407998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:36.408454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.410062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.433209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 398/799] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:36.437971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.437971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.437971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.437971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.438434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:36.438434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.438434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.439559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.441018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.452660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 402/800] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:36.456808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.561603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549287ba-adbd-4203-b20a-7a009a2bb3e8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.565744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.602191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.602831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.603113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.603388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.611480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.613005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:36.624378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:36.625408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:36.626394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 403/801] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:36.631244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.693096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.696698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d885469-e959-465f-bda8-0fceca66004e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.738203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674e3f3-dbbb-471b-b7b3-bcfd51ff1bd9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.743259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.743923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.744205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.744478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/7355daeb39aa4d8c8c1eb698fede46f4/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.746436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=7355daeb39aa4d8c8c1eb698fede46f4, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.747872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.761918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 399/802] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:36 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/7355daeb39aa4d8c8c1eb698fede46f4/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:36.776189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.777648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.778362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.778743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.779091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.779980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:36.780394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:36.781932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:36.812732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 404/803] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:36.814685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:36.901546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4576-8098-43b3-b4f6-afd58be37a18 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.914115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.954624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:36.955201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:36.955491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:36.955769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:36.959642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a82a-cfd0-4164-bd80-37a847045021 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:36.984411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:36.992328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:36.994605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:36.996662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:36.998172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:36.998319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 400/804] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:37.005140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.040306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.041008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.041292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.041729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.133708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.135690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800a158-2461-4fe7-9589-503caae97e2a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.168354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f21626-e43e-4b3c-a4e0-33b5f24dd09c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.207902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.208983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.209474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.209950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.210907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:37.211437 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.213120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa18687-968a-4ceb-8c7a-9054b3ca3177 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.222165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 401/805] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:37.224136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2429de-e785-4a4e-ae65-d016ea176a38 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.224657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2429de-e785-4a4e-ae65-d016ea176a38 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.225148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2429de-e785-4a4e-ae65-d016ea176a38 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.226678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 402/806] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:37.231635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2429de-e785-4a4e-ae65-d016ea176a38 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.256975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 405/807] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:39:37.261220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.261720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.262407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.270789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:37.289289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:37.290761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:37.302774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:37.362339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2429de-e785-4a4e-ae65-d016ea176a38 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.373028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 406/808] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:37.374060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.406369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.407312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.407602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.408142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.421088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.423338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:37.441905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:37.443158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:37.444130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:37.444130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:37.444130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:37.444130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 403/809] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:37.450676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.514756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53bb36-c9e2-4654-874a-ce8bb3f82389 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.588258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.588923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.589202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.589471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.616579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.616983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:37.618133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.625140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:37.655298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd6294-e1ea-4a3b-9878-82c6a775a9b4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:37.658859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:37.669520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:37.670428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:37.670428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:37.670428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:37.670428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 407/810] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:37.670428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.727352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.728056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.728300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.728594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.729302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:37.729609 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.731000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.743951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 404/811] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:37.748919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.816055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cd79a-e090-4a0b-b8c4-f31ea4a09ab1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.857672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.858328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.858655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.858867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.859435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:37.859741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.861213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.866872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17e4b8-87d8-4948-a341-591150451bef None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:37.874523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 408/812] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:37.875475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.920693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:37.921744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:37.922245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:37.922746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:37.923553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:37.925196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:37.938082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:37.939157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 20 04:39:37.940044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:37.947854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 405/813] 10.222.0.27 () {56 vars in 1129 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:37.947854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:37.951785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:37.965102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.001922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77027-5070-47bf-9e4c-96d8e7629de3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.040231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.040972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.041250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.041517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.048654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.050083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:38.063270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:38.064247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 409/814] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:38.065261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.078079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a4fef-9a47-49e2-99d3-e724aa3e29c1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.114680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.115592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.115886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.116162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.122863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.124301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-3ca00c4d-2a81-46cf-a0b2-ce22bb677f10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:38.136694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:38.137682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:38.138603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 406/815] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:38.139130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7650-9800-4101-833d-2e135476b359 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.139130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7650-9800-4101-833d-2e135476b359 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.139279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7650-9800-4101-833d-2e135476b359 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.140981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 407/816] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:38.141399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7650-9800-4101-833d-2e135476b359 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.197997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741119-e1e4-4ed9-bb9f-4b8785cecb36 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.214488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.217590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.239875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.240641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.240929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.241250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.241870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.242175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.243533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.253054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 410/817] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.253054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.258453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7650-9800-4101-833d-2e135476b359 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.295598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.296198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.296478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.296796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.297524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.297827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.299305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d61f2c-c5eb-4c4f-94db-27dcfe04f1f1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.322064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 408/818] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.322064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.322064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.322064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.340832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:38.342236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:38.361167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0ec56-509f-4363-8a49-e700190ada94 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.397250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.397944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.398291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.398625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.399421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.400011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.401518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.416629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 409/819] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:38.416629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.416871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 411/820] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.416871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.467776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.494288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.533211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e0a5e6-e93d-4fed-a834-ed4c9891d1a6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.550130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437094-896d-42f3-b98d-b472a246f3aa None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.569474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.570445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.570802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.571137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.585174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.587238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.592531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.593233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.593587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.593929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.595137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.595518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.597535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:38.603650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:38.604827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:38.605933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:38.605933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:38.605933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:38.606154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 410/821] 10.222.0.27 () {56 vars in 1124 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.610671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.611018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 412/822] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:38.611912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.727925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.737940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed27dff-70a0-480e-b81d-a8782c28cabb None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.739269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a460e3f-f6d2-4d07-93f7-d7a01630854d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.761161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:38.775121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.775727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.776048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.776272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.776933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.777234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.777770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.778379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.778591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.778688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.778917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.779639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:38.779940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.781416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.788088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 411/823] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.789441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.797757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 413/824] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:38.797757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:38.927987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a6e49-8d02-4feb-85b2-768f9189a54b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.947441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f65f-4408-4ae1-97a8-192c0580c90d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:38.966590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.967354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.967733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.968104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.968917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.970396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:38.993226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:38.994134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:38.995203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:38.995542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] PATH_INFO: `/v3/services/network` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:38.996228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:38.997810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.001537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 412/825] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:39:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:39.002885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:39.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:39.009371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:39.010234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 414/826] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:38 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:39.015603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.041542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.072508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.164524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bccfb-8a3f-4b03-8a11-13867480731c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.185765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4b2ed-8434-4687-ac37-0e4d15ce7714 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.207193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.208809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.209326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.209799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.217931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.219872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.230845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.231748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.232176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.232618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d6c0e397-6591-4429-9fbe-5e38b578644c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:39.232991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:39.233867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:39.234756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 413/827] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:39.236196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] RBAC: Adding query filter params (name=network) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:39.236311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7f311-bbf9-4ede-8d00-41e62e6a3694 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.236817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.238440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0678357-038b-4e3e-9afe-f0fd25139722 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.240877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7f311-bbf9-4ede-8d00-41e62e6a3694 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.241253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7f311-bbf9-4ede-8d00-41e62e6a3694 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.242800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 414/828] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:39.244699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 415/829] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:39.244812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7f311-bbf9-4ede-8d00-41e62e6a3694 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.252954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.253561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.253834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.276227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:39.277895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:39.363421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.371574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7f311-bbf9-4ede-8d00-41e62e6a3694 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.380635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 416/830] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:39.380635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.416989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.442829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.443852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.444334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.444840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.445802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:39.446318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.448058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.472629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 415/831] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:39.478962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.550860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91971e2b-cda5-4b51-856e-b4a702dc00a6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.594792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.595838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.596330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.596854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.597703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] RBAC: Adding query filter params (type=network) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:39.598882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.600643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.612745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 417/832] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:39.613959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.679274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8051d-a5b5-40af-a7d2-495a69aeb0db None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.691809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.719727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09432-4837-4f55-83d1-90908a291a56 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.731345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.731782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.732059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.732328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/aab2ee7790444760b8333226be499621/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.748291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=aab2ee7790444760b8333226be499621, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.749795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.759530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.760365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.760832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.761251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.762088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:39.763582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:39.766044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 416/833] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:39 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/aab2ee7790444760b8333226be499621/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:39.767156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:39.780607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:39.808901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 418/834] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:39:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:39.903330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf2870-961c-45d6-a12a-d76ea5724309 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:39.952047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:39.952714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:39.952993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:39.953261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:39.965603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:39.971207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:39.972385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:39.974110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 417/835] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:39.979164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.100304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b01e0-3027-4b8d-a52c-7110dc4b17ab None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:40.133778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:40.136878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:40.137496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:40.137783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:40.138097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:40.139043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:40.139343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:40.140838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:40.151554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 419/836] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:40.157255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.228479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:40.272773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e843-b73f-443c-a8ca-8121dd9b7f69 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:40.310518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:40.311517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:40.312021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:40.312589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:40.323970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:40.325760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:40.341101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:40.342079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:40.343006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:40.343006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:40.343006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:40.345139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 418/837] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:40.351284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.458681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcf85c-abaf-4833-8c3c-98f43de14a08 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:40.495308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:40.496279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:40.496794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:40.497262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:40.498083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:40.498598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:40.500190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:40.512987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 420/838] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:40.521681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.624237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d4202-6fe6-4f0a-8537-1e603040b1ab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:40.638405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:40.661604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:40.662221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:40.662499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:40.662769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:40.670192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:40.671610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:40.682756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:40.683675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:40.684655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 419/839] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:39:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:40.689737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.745902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:40.796077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6754764-9f3a-4fff-b45f-aabbbdc83e4b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:40.831892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:40.833117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:40.833395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:40.833661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:40.834381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:40.834679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:40.836091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:40.854048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 421/840] 10.222.0.27 () {56 vars in 1135 bytes} [Tue Sep 20 04:39:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:40.859705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:40.967245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c2a60-5fe0-4ea4-b3b4-32691b5eafb3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.003601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.004120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.004439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.004733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.052583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:41.079813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.081361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35b99c-b959-4bb4-a162-81075805528e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.183343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 420/841] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:39:41.188765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.306140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb6e45-7677-496c-9d02-b4a2eb19b22d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.317358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:41.342687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.343315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.343587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.343856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.355279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.356807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.368822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238543ae-50c1-4712-a8e9-5592cc0a9aa5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.369127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238543ae-50c1-4712-a8e9-5592cc0a9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.369371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238543ae-50c1-4712-a8e9-5592cc0a9aa5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.371053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 421/842] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:41.371955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-9e66-4aed-b8f8-4fd6e76f9cd4 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.372235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-9e66-4aed-b8f8-4fd6e76f9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c901603a-ce0d-4472-a116-673b023432c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:41.372407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:41.373389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:41.374832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:41.374832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:41.374832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:41.375043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbb1d1-9e66-4aed-b8f8-4fd6e76f9cd4 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.375127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 422/844] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:41.375208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 422/844] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:41.378135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8bc23-594c-4e08-a6ae-044198435cad None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.378419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8bc23-594c-4e08-a6ae-044198435cad None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.378695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8bc23-594c-4e08-a6ae-044198435cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.379149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbb1d1-9e66-4aed-b8f8-4fd6e76f9cd4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.396762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d6e8bc23-594c-4e08-a6ae-044198435cad None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:41.397963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d6e8bc23-594c-4e08-a6ae-044198435cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:41.461783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:41.472661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 423/845] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:41.472661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.472661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.472661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.490034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:41.491208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:41.510083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbb1d1-9e66-4aed-b8f8-4fd6e76f9cd4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.545747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.546580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.546726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.546982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.547667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:41.547970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.549496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.565510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 423/846] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:41.572714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.577850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 424/847] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:41.677260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dda526-8793-4bd5-b1a3-dd6decdd1755 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.711973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.712666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.712946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.713216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.715031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:41.720737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.722456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:41.734512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:41.735416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:41.736321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 424/848] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:41.742341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.743028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.773773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:41.856856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b74bc-3b03-4a79-8bfc-a5152135658e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.870040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66634d-9e12-47ae-b65b-817649f22f2a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:41.894420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.895096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.895366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.895630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.896200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.897608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.903965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:41.904567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:41.904876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:41.905156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:41.905842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:41.906147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:41.906562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:41.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:41.908948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:41.909703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:41.909774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 425/849] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:41.909774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.930135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 425/850] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:41.930135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:41.983890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:42.007687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:42.023447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df415b61-b182-4179-a874-2aa1b9241163 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.039994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378fb1-6235-471d-bbf8-bd1d9c68248a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.061072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.061700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.061979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.062248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.062821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.064169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:42.073541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:42.074584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:42.075478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 426/851] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:42.076787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:42.087254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.088245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.088627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.088920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.089861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:42.090134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.091493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.101037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 426/852] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:42.101037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:42.183331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e213-ebf3-4cf7-96be-34f1b4c493c8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.213319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa7ad0-63b3-4334-b0fe-6c50d22d53d1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.218527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.219135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.219417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.219689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.220408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=7355daeb39aa4d8c8c1eb698fede46f4) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:42.220749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.222328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.230593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 427/853] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/role_assignments?user.id=7355daeb39aa4d8c8c1eb698fede46f4&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:42.231870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:42.250925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:42.253280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:42.267040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.268190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.268827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.269290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.270713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:42.271210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.273152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.279676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 427/854] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:42.284957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:42.356994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebb3dd-e0f7-495d-ae5b-450b4b0fcb19 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.405821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.406585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256e837-89c4-47f1-8c75-2a0f597d08a5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.406766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.407193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.407624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.408718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:42.409163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.411743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb8d23-230b-439d-8002-da537bab5270 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.419440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 428/855] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:42.421287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb4877-cf18-4b81-9537-8ffe87b1323b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.421752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb4877-cf18-4b81-9537-8ffe87b1323b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.422268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb4877-cf18-4b81-9537-8ffe87b1323b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.424445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 429/856] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:42.431181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.431458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.431727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.452224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:42.453478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:42.461396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.462402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.462861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.463369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.464217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:42.464573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:42.466052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:42.471534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 428/857] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:42.537752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 430/858] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:42.827183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:42.877304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:42.934365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860b06-9dc5-49d0-9cbc-5dfcb2fab54d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:42.970146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:42.971152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:42.971631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:42.972100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:42.983358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:42.989488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:42.990456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:39:42.991324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:42.994003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 429/859] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:43.003226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.010200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:43.111376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca581d-d93d-49b6-a44b-6a9bcddcfa11 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:43.146998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:43.147606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:43.147890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:43.148160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:43.148851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:43.149152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:43.153335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:43.160344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 431/860] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:43.166476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.275467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4e9cf-0e3e-4cdc-afcb-994c2ee1870e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:43.283198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:43.313923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:43.314839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:43.315298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:43.315751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:43.326580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:43.328512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:43.344122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:43.345124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:43.346024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:43.346024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:43.346024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:43.347802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 430/861] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:43.354307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.433259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:43.467718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123dbabc-a929-4b21-80d0-b5b472141df7 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:43.503585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:43.504439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:43.505056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:43.505475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:43.506247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:43.506696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:43.508193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:43.521448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 432/862] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:43.527315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.630103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e048872-e1db-4aa5-a3ae-6821150ae94e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:43.665588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:43.666605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:43.667136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:43.667608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:43.675428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:43.677180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:43.688865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:43.689820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:43.690818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:43.693852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 431/863] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:43.699641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.807296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71e02-b242-4203-82fc-12c28fad1944 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:43.837825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:43.844422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:43.845293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:43.845733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:43.846182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:43.847059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:43.847670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:43.849953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:43.872715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 433/864] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:43.874638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:43.894676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:43.981913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca716d-3a79-46c6-906c-9f140b9aef87 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:44.016390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:44.017021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.017302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.017571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.086760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:44.088222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:44.109531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:44.202975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 432/865] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:39:44.209461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.314688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6a659-1840-470f-85f1-7787e3a845c3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:44.352086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:44.353136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.353663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.354133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.365013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:44.366725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-31f24f3f-6073-48f3-80d3-94cdc6cb9a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:44.381694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:44.382691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:44.383595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:44.383595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:44.383595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:44.385626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 434/866] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:44.391413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.500477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb618d-2768-4bcc-8082-b8f11030bb56 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:44.522557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:44.540647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0de6-1402-4a38-822d-2f8d228337bc None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.540647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0de6-1402-4a38-822d-2f8d228337bc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.540647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f0de6-1402-4a38-822d-2f8d228337bc None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.542069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 435/867] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:44.543223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:44.543852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.544131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.544401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.545080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:44.545447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:44.546983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d52f8a-df70-4bd1-a43f-767ec3dfd5e9 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.547182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:44.547296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d52f8a-df70-4bd1-a43f-767ec3dfd5e9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.547515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d52f8a-df70-4bd1-a43f-767ec3dfd5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.561086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 433/868] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:44.566499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.571641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-00d52f8a-df70-4bd1-a43f-767ec3dfd5e9 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:44.572899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-00d52f8a-df70-4bd1-a43f-767ec3dfd5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:44.620272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:44.663139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 436/869] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:44.685272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c35b0-3350-4439-8e4a-b4350a11c825 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:44.699696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59d001-55b5-4f27-8396-2b2916d17989 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.700199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59d001-55b5-4f27-8396-2b2916d17989 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.700305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59d001-55b5-4f27-8396-2b2916d17989 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.701844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 437/870] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:44.707132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75221-ad2f-43d5-b8ef-c005e5c13653 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.707971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75221-ad2f-43d5-b8ef-c005e5c13653 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.708238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75221-ad2f-43d5-b8ef-c005e5c13653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.726875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:44.727533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.727814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.728083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.728722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b1b75221-ad2f-43d5-b8ef-c005e5c13653 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:44.729957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b1b75221-ad2f-43d5-b8ef-c005e5c13653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:44.737628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:44.739858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:44.757536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:44.759334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:44.760856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 434/871] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:44.765216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.821255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:44.824132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 438/872] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:44.852116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea4f9-f9ac-4261-912a-b9c0997442ca None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.852656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea4f9-f9ac-4261-912a-b9c0997442ca None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.853079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea4f9-f9ac-4261-912a-b9c0997442ca None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.858335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 439/873] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:44.859915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.860355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.860815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.880156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:44.881572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:44.888512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0aee0-4b9e-4a6c-87eb-5b4cda34f230 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:44.935603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:44.936246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:44.936478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:44.936784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:44.937470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:44.937780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:44.939878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:44.952428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 435/874] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:44.953418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 440/875] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:44.958096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.961840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:44.993786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:45.073159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:45.074304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58098c-8e3a-439f-bc22-a12aef1015fe None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.078698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f758a3-7c4b-4d23-8e89-d9ce41e6af7f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.112582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.113226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.114034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.114350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.115035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=aab2ee7790444760b8333226be499621) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:45.115347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.116147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.116805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.116972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.117424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.117888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.118923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.121496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.123364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 441/876] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/role_assignments?user.id=aab2ee7790444760b8333226be499621&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:45.125369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:45.146816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:45.148888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:45.152251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 436/877] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:45.162008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:45.248930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:45.259315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ee852-a118-4653-b344-735f0bbd4bae None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.298971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.299591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.299868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.300141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.300819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.302183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.313135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8ee89-aed4-4225-b425-43bd629fbd39 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.324721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 442/878] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:45 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:45.324721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:45.373760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.374385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.374691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.374967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.375613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:45.375919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.377623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.383539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 437/879] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:45.389256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:45.422598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:45.491828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff52f-b450-45a8-a93d-771cfbcdb02d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.528978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.529746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.529962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.530242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.530895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.532352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0edd41-4fb6-4193-9b36-e031efe24a58 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.540595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc6e6-b5d7-4c97-a921-2d06448db64f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:45.557194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 443/880] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:45 2022] POST /identity/v3/services => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:45.589624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:45.590165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:45.590442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:45.590712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:45.591470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:45.591771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:45.593208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997a5e-3b8a-4b36-a708-f0018e8d4107 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:45.599675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 438/881] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:47.512093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2574-7d8e-46a5-a4ac-43158cb4d827 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.512639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2574-7d8e-46a5-a4ac-43158cb4d827 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.512639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2574-7d8e-46a5-a4ac-43158cb4d827 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.514236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 444/882] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:47.519727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba65b-37c4-449c-ad06-93517b29ce36 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.520156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba65b-37c4-449c-ad06-93517b29ce36 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.520336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba65b-37c4-449c-ad06-93517b29ce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.539591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-039ba65b-37c4-449c-ad06-93517b29ce36 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:47.540885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-039ba65b-37c4-449c-ad06-93517b29ce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:47.566242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789bd9-e564-4435-9c7e-fab83b70ef02 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.566589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789bd9-e564-4435-9c7e-fab83b70ef02 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.566819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789bd9-e564-4435-9c7e-fab83b70ef02 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.568455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 445/883] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:47.573636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87957-2829-4ae3-b85e-de043cdece04 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.574023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87957-2829-4ae3-b85e-de043cdece04 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.574399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a87957-2829-4ae3-b85e-de043cdece04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.593876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-67a87957-2829-4ae3-b85e-de043cdece04 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:47.595752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-67a87957-2829-4ae3-b85e-de043cdece04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:47.610106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:47.624495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 439/884] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:47.679128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 446/885] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:47.802867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b2ba6-c4b6-4dd6-96c5-96e87af85024 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.803170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b2ba6-c4b6-4dd6-96c5-96e87af85024 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.803355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b2ba6-c4b6-4dd6-96c5-96e87af85024 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.804936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 440/886] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:47.810080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.810366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.810638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.815934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d64160-bf1f-477b-be00-09bf89079c8b None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.816214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d64160-bf1f-477b-be00-09bf89079c8b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.816533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d64160-bf1f-477b-be00-09bf89079c8b None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.818588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 441/887] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:47.824666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:47.825208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:47.825697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:47.831622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:47.833080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:47.843856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:47.845285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:47.853281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:47.915945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 447/888] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:47.916351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 442/889] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:47.916827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:47.975376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:47.998725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.028272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4e75a-30d6-454e-9c44-458f510a6ac0 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.067152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.067715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.067992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.068261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.080058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:48.085394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:48.086342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:48.087229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:48.088522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 448/890] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:39:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:48.088633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70bbd-4be6-4f4a-9648-332148d6478c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.095569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.131167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.131167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.131167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.131167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.131566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.132720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:48.142319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:48.143287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.145170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 443/891] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Sep 20 04:39:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:48.149459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.161420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.234779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bad045-2edd-42fc-8f68-33938394d441 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.262833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa44b8-2087-482e-b823-d0c5157545f6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.268891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.269504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.270394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.270692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.271345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.271648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.273068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.274401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.282012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 449/892] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.282784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.298068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.298675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.298961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.299234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.299860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.300161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.301584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.306290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 444/893] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.307184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.374211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.387768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9700-811c-4b9d-88ed-6fa618b97a75 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.405365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd351632-66a9-45ee-9c81-67849e27dec5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.420162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.420807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.421081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.421350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.421983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.423325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.437178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.437364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 450/894] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:48 2022] POST /identity/v3/services => generated 241 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:48.437812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.438032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.438130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.438333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.438900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.440254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:48.450687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:48.451820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:48.458528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 445/895] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:48.459552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.518740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.575673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9117cde-74f4-4f9b-aecc-9a3c4b9a0f3f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.577076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11c484-98c3-460c-8710-c05341283f06 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.612926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.613568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.614481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.614481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.614757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.615060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.616308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.616455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.617003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.618476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.618476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.618646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.623471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 446/896] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.624554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.632303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.634667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:48.651544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:48.652587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:48.653901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:48.653901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:48.653901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:48.653901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 451/897] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:48.653901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.743534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c31ec-7664-466a-8a3a-7e6b5b4b2baf None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.759326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b6032-f53f-4474-9985-3bd848e6e439 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.761610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.780320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.780952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.781231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.781923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.782550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.783118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.784950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.796344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 447/898] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.796344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.799799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.800407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.800708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.801056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.801747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] RBAC: Adding query filter params (interface=public, service_id=adff1eb894994808b864583a77180444, region_id=RegionOne) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.802046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.803397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.813122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 452/899] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/endpoints?service_id=adff1eb894994808b864583a77180444&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.813122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.862953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:48.907670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2cde-fd30-4015-87e1-837c3d177288 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.922556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5fc6-de58-4b82-9842-3ad59d20a38a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:48.950044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.950656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.951050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.951326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.952199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.952507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.954018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.959881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:48.960536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:48.960843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:48.961113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:48.961731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:48.962033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:48.963379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:48.971787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 453/900] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.971787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:48.972058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 448/901] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:48.973431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.003200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.076581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552e3c-5959-4901-abdf-3d14cfbd6059 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.080156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425592e7-b10c-4934-ad4b-3bde10e031c7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.107788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.111158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.111804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.112084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.112356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.112993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.114982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.128637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 454/902] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:39:49.128637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.130258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.130915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.131190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.131461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.140803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.142542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:49.156553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:49.157760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:49.158737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 449/903] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:49.164091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.254923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.262613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac7dd-16e7-4958-ab26-42ea8b0fe09e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.285872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309b0bb-bf94-46a6-a15b-267f2b6e0e9b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.310929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.312047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.312499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.312939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.313796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] RBAC: Adding query filter params (interface=public, service_id=4e04ed940f1e4192bcb56de2b6b976af, region_id=RegionOne) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:49.314304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.316290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.324204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 455/904] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/endpoints?service_id=4e04ed940f1e4192bcb56de2b6b976af&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:49.327198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.327830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.328115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.328387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.330100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.330720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:49.331156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.333210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.358427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 450/905] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:49.359777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.412031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.459753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce0150-5504-443c-9614-de0baa726951 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.477083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6d152-9fc5-49d3-a563-a579c2c0b036 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.510552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.511673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.512506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.513079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.513933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.515752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.517221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21feaff-259c-486a-a459-ba346267feec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.517812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21feaff-259c-486a-a459-ba346267feec None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.518104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21feaff-259c-486a-a459-ba346267feec None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.518490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21feaff-259c-486a-a459-ba346267feec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.529247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 456/906] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:39:49.597454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21feaff-259c-486a-a459-ba346267feec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.598984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21feaff-259c-486a-a459-ba346267feec None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.644423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.724869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 451/907] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:39:49.726943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:49.756285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:49.838333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203243e-3122-41e7-bdc1-c279c1aa0b66 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:49.873930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:49.874565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:49.874845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:49.875119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:49.886059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:49.887506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:49.902519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:49.903559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:49.904472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:49.904472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:49.904472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:49.908306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 457/908] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:49.909752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21feaff-259c-486a-a459-ba346267feec None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:50.018491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21feaff-259c-486a-a459-ba346267feec None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:50.056833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:50.057465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.057747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.058022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.058698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:50.059002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:50.060514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:50.070229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:50.074863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 452/909] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:50.080907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:50.169672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:50.194049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1cde8-b07b-4de4-9c59-73e5a4e47686 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:50.229701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:50.230455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.230741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.231001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.237836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:50.239299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:50.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:50.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:50.252710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 458/910] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:50.259473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:50.367915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60969ee-35fb-4e68-8717-e1740a2b3efb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:50.404580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:50.405603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.406090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.406567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.407505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:50.408015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:50.409743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a34503-46ba-4669-83e2-b90db96bae82 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:50.428177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 453/911] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:50.429804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:50.540698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ea134-3270-4afc-927a-dedc968faea6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:50.581907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:50.583192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.583470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.583771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.584498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] RBAC: Adding query filter params (role.id=a0524931c0d2413d955919535ffeecaa, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=aab2ee7790444760b8333226be499621) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:50.584827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:50.586247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361119d8-24c1-4948-8841-8832fce0d2bd None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:50.589112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:50.592519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 459/912] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:39:50 2022] GET /identity/v3/role_assignments?user.id=aab2ee7790444760b8333226be499621&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=a0524931c0d2413d955919535ffeecaa => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:50.668103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406500f8-875c-47b2-892f-c490d0bd89be None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.668656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406500f8-875c-47b2-892f-c490d0bd89be None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.668656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406500f8-875c-47b2-892f-c490d0bd89be None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.670214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 454/913] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:50.675875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:50.676171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:50.676432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:50.696416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:50.697720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:50.773275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 460/914] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:51.041571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406500f8-875c-47b2-892f-c490d0bd89be None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.105411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:51.150207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406500f8-875c-47b2-892f-c490d0bd89be None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.186490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.187122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.187402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.187676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.188258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.189776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-1717b12c-2855-41b9-9b4c-871c087689ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:51.199482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:51.200482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.201749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 455/915] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:51.206812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.315899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0083a4a-b99c-48b6-8744-0280e895979a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.342189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa8d9-074f-4509-86c7-d15bfc701bfc None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.342497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa8d9-074f-4509-86c7-d15bfc701bfc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.342720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa8d9-074f-4509-86c7-d15bfc701bfc None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.344300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 456/916] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:51.348298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:51.349286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.349507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.349787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.353211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.353832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.354109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.355125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.355125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:51.355425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.356657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.362084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 461/917] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:51.369378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:51.369590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.370563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:51.450693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 457/918] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:51.501007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8738e3-69f8-4398-971f-798d331457f3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.545525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.546287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.546468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.546636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.547338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:51.547639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.549093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.555464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 462/919] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:51.561342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.621347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:51.665817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13255f93-5fbe-4136-bd5e-3e63440b6632 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.699700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.700450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.700636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.700908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.701613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] RBAC: Adding query filter params (interface=public, service_id=7bfa7034ff524d31b5a94286007af8a5, region_id=RegionOne) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:51.701919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.703278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.711446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 458/920] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/endpoints?service_id=7bfa7034ff524d31b5a94286007af8a5&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:51.717191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.725619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.792445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:51.835343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a87a59-1f36-4c9d-a279-b6d61783ecab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.850078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a6cd-3939-4356-b9fd-9192b019f142 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:51.855460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:51.875128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.876180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.876728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.877111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.877821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.879874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:51.889213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:51.890435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 20 04:39:51.891565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:51.893747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 459/921] 10.222.0.27 () {56 vars in 1132 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:51.897490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:51.898371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.898850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.899317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.901261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:51.902754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5138e-9150-4b5c-be54-9bfd5f70a177 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:51.905652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:51.909684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 463/922] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:39:51.922988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b14e-b228-4d4d-927a-06661b4c1016 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.923311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b14e-b228-4d4d-927a-06661b4c1016 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.923545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1b14e-b228-4d4d-927a-06661b4c1016 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.929944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 464/923] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:51.931639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:51.931918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:51.932185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:51.954304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:51.955532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:52.037928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 465/924] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:52.038930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19617-0aab-4b10-8b9a-ed0efb576b7a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.078492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.079116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.079396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.079665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.080306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:52.080629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.082056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.087821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 460/925] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:52.093527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.151489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:52.202406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92209d2-1b8f-4f1e-a938-1a891b2e66e2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.236358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.236994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.237267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.237531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.238774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:52.239065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.240423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.245710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 466/926] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:52.251266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.299303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:52.313592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.363586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a5639-341c-425e-a898-4a50bbef6762 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.410758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.411680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.411967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.412237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.412949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.414852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.426553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1f2ec-ce54-4058-9786-c78598fe20d8 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.442314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 461/927] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:39:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:39:52.449381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:52.481480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.482138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.482415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.482685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.483337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.484825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:52.494527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:52.495588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:52.497771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 467/928] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:52.506094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.624914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372fe9f-1c25-4157-96e9-4bade080e607 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.626514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:52.664766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.665299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.665581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.665852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.666478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:52.666777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.668137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.673104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 462/929] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:52.679149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.793993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473db91-36e4-46ed-b2aa-a4f9f6a5870a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:52.834082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:52.835085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.835571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.836046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.836943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:52.837472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:52.839063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633a3c9-6226-4dda-a900-96a428ff730a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:52.845142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 468/930] 10.222.0.27 () {56 vars in 1114 bytes} [Tue Sep 20 04:39:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:52.851591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:52.933845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2fd1d-aa87-49c4-b09f-c40ddb08739b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.934037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2fd1d-aa87-49c4-b09f-c40ddb08739b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.934343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2fd1d-aa87-49c4-b09f-c40ddb08739b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.936978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 469/931] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:39:52.940908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:52.941191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:52.941498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:52.963913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:52.966229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:52.971236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c73fa8-e680-43cc-a374-8c59e6d16a97 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:53.007492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:53.007966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:53.008317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:53.008711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:53.009473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:53.010926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:53.025519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:53.040294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 463/932] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:39:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:53.062231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 470/933] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:39:53.358401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:53.384316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:53.471637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d9250-647a-4725-b22a-9cbb8e35162c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:53.509277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:53.510260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:53.510744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:53.511233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:53.522012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:53.527298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:53.528260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:53.529192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:53.531761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 464/934] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:39:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:53.538305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:53.644833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba4967-fa97-46dc-b096-d5340c814f14 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:53.680482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:53.681919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:53.682409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:53.682886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:53.683734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:53.684228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:53.685801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:53.700671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:53.701801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 471/935] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:39:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:53.790244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:53.811116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345dbb-de2f-4ae2-9552-09f09d940bf2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:53.851897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:53.853367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:53.853869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:53.854335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:53.865383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:53.867108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:53.882335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:53.883429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:53.884345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:53.884345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:53.884345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:53.886454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 465/936] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:53.892275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:53.929059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.003579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d3de9-e614-419b-8bc5-2bbb4204ae9c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.048637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.048637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.048637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.048637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.049025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:54.049025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.049025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edca2-534f-4cf8-b717-f9bc9b49ea91 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.064585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 472/937] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:54.067082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.188726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd05b96-810c-4af4-8c04-09ac36dac981 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.191952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaad775-f7bf-4b7b-ad8a-93300aada36a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.192450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaad775-f7bf-4b7b-ad8a-93300aada36a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.192988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaad775-f7bf-4b7b-ad8a-93300aada36a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.194865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 473/938] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:54.200066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.200592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.201092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.219666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:54.221182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:54.226769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.227409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.227692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.227968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.229920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.236571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.239194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:54.251617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:54.252632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:54.253605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 466/939] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:54.260558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.308977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 474/940] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:54.368016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf1f8-b9ba-4975-bdf0-78c7f74c9527 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.403857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.404521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.404834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.405158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.405901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:54.406210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.407723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.426879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 467/941] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:54.434021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.453719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.537068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8da1e-3f3e-466e-a174-f2d4685a7ebb None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.571828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.572794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.573290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.573755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.584038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.585754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:54.606663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:54.607740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:54.608669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:54.608669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:54.608669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:54.615330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.615453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 475/942] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:54.616318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.683918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.713828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.748058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395c95-4c0e-4d6d-bac9-d670e855e05a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.756521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1176b7-855b-4946-baad-035388b91257 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.791494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.792122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.792398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.792709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.793289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.794643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.799929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.800855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.801328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.801828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.802643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:54.803100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-64cb9829-fc56-4367-8e6b-9ce9e0055ce3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:54.803305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:54.804419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:54.805307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 468/943] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:54.806102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d819d-e9d7-45db-87a9-91fe45f94e5a None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.806102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d819d-e9d7-45db-87a9-91fe45f94e5a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.806318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d819d-e9d7-45db-87a9-91fe45f94e5a None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.807735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6b7d4-b722-4af1-a4e7-5880d7c4dd22 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:54.811572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 469/944] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:54.812351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d819d-e9d7-45db-87a9-91fe45f94e5a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.826687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 476/945] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:54.826687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.826687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.826687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.844353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:54.845546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:54.919804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 477/946] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:54.920492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:54.936802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d819d-e9d7-45db-87a9-91fe45f94e5a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:54.963395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.982801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:54.990798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:54.992186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:54.992699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:54.993412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:54.994216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:54.994937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:54.997383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.004243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 470/947] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:55.013582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.029894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fe8a6-736b-433a-b920-e176289eddd6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.072505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.073621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.074017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.074442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.082490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.084662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:55.097530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:55.098428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:55.099358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:55.099793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 478/948] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:55.106643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.143887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739104a2-3955-4176-9dd2-b748aaa8e7e4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.198996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.200321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.200909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.202460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.203372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:55.203920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.205744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.214055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 471/949] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:55.214235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.237077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b337e-6bee-4691-8eb4-83e3c1e26eab None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.258625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:55.262400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:55.277269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.277932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.278310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.278633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.279324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:55.279628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.281295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.299543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 479/950] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:55.300398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.321508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3f02-3739-4da7-afd1-f0f22173198c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.353699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.354319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.354601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.354881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.355455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.356855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:55.366068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:55.367067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:55.367913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 472/951] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:55.368772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.401641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6e2a6-5a24-4048-9c78-e1824ea899ff None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.435329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.435936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.436215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.436486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.437176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.438536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff77d3-d1da-4888-800c-b9efbb3a86ab None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.457779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 480/952] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:39:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:55.458964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0fd5-50d9-48ae-b6f2-012186afe848 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.459246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0fd5-50d9-48ae-b6f2-012186afe848 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.459515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b0fd5-50d9-48ae-b6f2-012186afe848 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.460739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 481/953] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:55.461438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b0fd5-50d9-48ae-b6f2-012186afe848 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.483994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b80cc-f9ac-4c6e-aec6-c419113bafe5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.495988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:55.506103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:55.532882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.533553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.533829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.534102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/aab2ee7790444760b8333226be499621/roles/a0524931c0d2413d955919535ffeecaa` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.536781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=aab2ee7790444760b8333226be499621, role_id=a0524931c0d2413d955919535ffeecaa)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.538242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb88cee-2386-4cf9-b825-f955743a0f5c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.554300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 473/954] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:39:55 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/aab2ee7790444760b8333226be499621/roles/a0524931c0d2413d955919535ffeecaa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:39:55.556323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.556874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.557606 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.580589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:55.582396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:55.606201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b0fd5-50d9-48ae-b6f2-012186afe848 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.651327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.651948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.652226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.652496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.653187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:55.653482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.654878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.663951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 482/955] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:55.670415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:55.689455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 474/956] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:55.799394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:55.803254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2a11a-fbcd-4b21-9749-587790651364 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:55.839303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:55.839912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:55.840186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:55.840450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:55.841108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:55.841402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:55.842757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:55.848554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 483/957] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:55.962021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:56.064988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:56.072573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c80de9-308c-42fa-bb7f-13a32185f283 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:56.108926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:56.109540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:56.109816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:56.110086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:56.110745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:56.112181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:56.115445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING keystone.server.flask.common [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 20 04:39:56.137695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.sql.core [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 20 04:39:56.137695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 20 04:39:56.137695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [parameters: {'id': '5d65c03c325f4d719003405cfe9a56bd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 20 04:39:56.137695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87798) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5d65c03c325f4d719003405cfe9a56bd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5d65c03c325f4d719003405cfe9a56bd. Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:56.155807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 20 04:39:56.156647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application yield resource Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self.session.rollback() Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self.force_reraise() Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise self.value Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application session.commit() Sep 20 04:39:56.157831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._prepare_impl() Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self.session.flush() Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._flush(objects) Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application compat.raise_( Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application flush_context.execute() Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 20 04:39:56.158756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rec.execute(self) Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 20 04:39:56.159670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:56.162503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [parameters: {'id': '5d65c03c325f4d719003405cfe9a56bd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:56.164687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:56.166670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5d65c03c325f4d719003405cfe9a56bd. Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:56.173041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 475/958] 10.222.0.27 () {60 vars in 1140 bytes} [Tue Sep 20 04:39:55 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:56.177197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:56.284882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d2ef-3160-4917-8fee-da24a82423e6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:56.320524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:56.321160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:56.321444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:56.321811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:56.328869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:56.330388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-64a0fff1-ad40-4734-9b1c-efaefe178b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:39:56.342728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:56.344623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:56.345703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 484/959] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:56.351612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:56.461675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a213-69d3-46fe-864b-5287aff794a1 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:56.496087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:56.496735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:56.497016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:56.497287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:56.497942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:56.498241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:56.499866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf66b0f-ddc1-4889-ad59-c9e06b030170 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:56.521067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 476/960] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:39:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:39:57.732367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db026-5f8d-450f-bfcd-3647d7016eba None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:57.733229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db026-5f8d-450f-bfcd-3647d7016eba None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:57.733229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5db026-5f8d-450f-bfcd-3647d7016eba None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:57.734570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 485/961] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:57.740341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795059-6089-4dbe-afcf-7999e26a0ce6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:57.740934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795059-6089-4dbe-afcf-7999e26a0ce6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:57.740934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795059-6089-4dbe-afcf-7999e26a0ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:57.761610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-22795059-6089-4dbe-afcf-7999e26a0ce6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:57.763196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-22795059-6089-4dbe-afcf-7999e26a0ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:57.767678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:57.840077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 477/962] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:57.873140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c606-e51e-427a-ae1b-ef15b125787d None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:57.873507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c606-e51e-427a-ae1b-ef15b125787d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:57.873850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5c606-e51e-427a-ae1b-ef15b125787d None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:57.875473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 486/963] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:57.881298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b11a-e09c-4b2c-b923-4bb3f4d94072 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:57.881583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b11a-e09c-4b2c-b923-4bb3f4d94072 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:57.881925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70b11a-e09c-4b2c-b923-4bb3f4d94072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:57.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ec70b11a-e09c-4b2c-b923-4bb3f4d94072 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:57.902136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ec70b11a-e09c-4b2c-b923-4bb3f4d94072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:57.981012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 478/964] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:58.030888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9854-210e-4042-a7ae-594d62a1db56 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.031211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9854-210e-4042-a7ae-594d62a1db56 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.031473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9854-210e-4042-a7ae-594d62a1db56 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.033178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 487/965] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:58.041985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.041985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.041985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.057942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:58.059298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:58.123472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.128644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 479/966] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:58.130588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca9854-210e-4042-a7ae-594d62a1db56 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.161707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.246296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca9854-210e-4042-a7ae-594d62a1db56 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.279924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.280628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.280972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.281234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.281815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.283175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.292148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:58.292704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:58.293834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:58.294782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 488/967] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:58.299616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.392578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.406421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452534-a907-4986-bef6-93ea59a9cf1a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.419771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48bf12-17c4-496f-b660-93e2d5d46f6b None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.443742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.444847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.445189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.445762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.446397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:58.446704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.448071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.453923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 489/968] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:58.454739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.463607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.464242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.464517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.464816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.475717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:39:58.480561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:58.481664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 480/969] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:58.483191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.557295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.570195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caa81b-2c55-4ec5-9bd2-f0a8f9e8e727 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.605711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25b893-562e-47df-aad7-fe63ddf8443a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.610988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.611596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.611871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.612138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.612842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.614627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.632810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 490/970] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:39:58 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:39:58.633637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.651091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.669371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.670521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.671415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.672181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.673078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:58.673822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.676574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.689085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 481/971] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:58.691732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.772064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861ac2b-0ecf-48a3-bf57-28cb1af8318f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.818989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.819719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.820112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.820474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.821369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:58.821650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.823114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be0e26-5df0-4fae-bff5-ba92350e4cca None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.832458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf164f86-8bc0-4125-95b3-b7afe5e6e094 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.835345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 491/972] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:58.836453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c376bc-5d22-47e6-857d-9973aee3ce71 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.837499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c376bc-5d22-47e6-857d-9973aee3ce71 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.837499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c376bc-5d22-47e6-857d-9973aee3ce71 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.838207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 492/973] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:39:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:39:58.840619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c376bc-5d22-47e6-857d-9973aee3ce71 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:58.859301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.875632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:58.876233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.876514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.876852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.877565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] RBAC: Adding query filter params (interface=admin, service_id=7bfa7034ff524d31b5a94286007af8a5, region_id=RegionOne) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:58.877880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:58.879776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8aa227-28a3-4539-9c54-172e01e1eb29 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:58.888671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 482/974] 10.222.0.27 () {56 vars in 1249 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/endpoints?service_id=7bfa7034ff524d31b5a94286007af8a5&interface=admin®ion_id=RegionOne => generated 193 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:58.889958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:58.890950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:58.891272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:58.910632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:39:58.911973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:39:58.932819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:58.961625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c376bc-5d22-47e6-857d-9973aee3ce71 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:58.988633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 483/975] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:39:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:39:58.988633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.006913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.007664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.007946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.008216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.022318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.024270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:39:59.044324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:59.045684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:39:59.046944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:39:59.046944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:39:59.046944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.048743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 493/976] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:59.052017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.095872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695e14-3f63-449d-a624-2838a4884db8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.111830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.145267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.146148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.146598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.147039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.147813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.149859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.168635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 484/977] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:39:58 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:39:59.188467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db9cda-7ea9-43f6-b594-f6451cc79113 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.218935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.229506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.230126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.230405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.230677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.231343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:59.231695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.233201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.247271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 494/978] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:59.253016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.276190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.371348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9dcfe-480a-426f-b859-3843ce1992b8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.408088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb4a7-1a8b-423e-9d1c-a74c8f31a62a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.419038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.419706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.420427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.420746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.431503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.433525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:59.448733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:59.449826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 20 04:39:59.450908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:39:59.453365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 485/979] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:59.454116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.454797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.455149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.455516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.456235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.458075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.460679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING keystone.server.flask.common [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 20 04:39:59.461742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.475723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.sql.core [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 20 04:39:59.475723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 20 04:39:59.475723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [parameters: {'user_id': '57d42369d7e044cda3dec549911084d7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 20 04:39:59.475723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87797) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:59.484477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:59.485468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application yield resource Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application next(self.gen) Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.session.rollback() Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.force_reraise() Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise self.value Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application session.commit() Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 20 04:39:59.486548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._prepare_impl() Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.session.flush() Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._flush(objects) Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application compat.raise_( Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application flush_context.execute() Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rec.execute(self) Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 20 04:39:59.487550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 20 04:39:59.490700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result = self._query(query) Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application conn.query(q) Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application result.read() Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 20 04:39:59.492841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [parameters: {'user_id': '57d42369d7e044cda3dec549911084d7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:59.494928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 20 04:39:59.497166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 495/980] 10.222.0.27 () {60 vars in 1131 bytes} [Tue Sep 20 04:39:59 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:39:59.500748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.532168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.546685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.604861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18b15a-085a-465a-a858-6000e3d4db9d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.617434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388e6f-ec0b-4dd9-95a4-29c2cc689300 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.642073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.642773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.643141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.643502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.652191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.654102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.663083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.663869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.664313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.664778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.665656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:59.666131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:39:59.666527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:39:59.667663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 20 04:39:59.668884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:39:59.670817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.675171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 496/981] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:39:59.676151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.698785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 486/982] 10.222.0.27 () {56 vars in 1149 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:59.705600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:39:59.810721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d57e6b-96fb-4f36-b3bc-9fee30b9da45 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.824257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.828639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:39:59.836686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b085d-d12f-44a9-ab9a-7f6d1888992f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:39:59.860669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.861520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.862215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.862642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.864682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:39:59.865153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.867422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:39:59.874854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:39:59.875579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:39:59.875963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:39:59.876337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:39:59.919241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 497/983] 10.222.0.27 () {56 vars in 1101 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:39:59.963511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:39:59.965022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.071325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:00.079739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 487/984] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:39:59 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:00.085844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:00.204486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39c915-47f4-4635-a686-0cc6a87f6ebf None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:00.238510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:00.239147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.239434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.239690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.250653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:00.252118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:00.270419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:00.271456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:00.272373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:00.272373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:00.272373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:00.272373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 498/985] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:00.277424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:00.375082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167c7c6-e316-4dd2-bf93-81e6423d6e15 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:00.409526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:00.410154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.410432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.410705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.411340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:00.411647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:00.413036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.428554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 488/986] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:00.429700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:00.449507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:00.533553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45f66-4e5d-41d7-8f45-091a142f1661 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:00.567290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:00.567912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.568189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.568455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.575091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:00.576539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:40:00.588135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:00.589169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:00.590096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 499/987] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:00.599080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:00.668261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:00.715050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618d46-c6bb-455d-9a9c-d3878e2ffcab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:00.749777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:00.750413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.750694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.750966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.751696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:00.752017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:00.753563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15aa82a-52ab-4374-8417-4cf2ec676fb8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.772649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 489/988] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:00.774412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:00.855832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:00.870373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18c7f-003d-44ee-b065-7e34eea299a0 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.870872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18c7f-003d-44ee-b065-7e34eea299a0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.870985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18c7f-003d-44ee-b065-7e34eea299a0 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.872363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 490/989] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:00.876970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.877250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.877520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.879270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf37436-6ee4-4856-a775-5e2d7ce46609 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:00.895646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:00.896847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:00.922443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:00.923210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:00.923589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:00.923963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:00.924842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] RBAC: Adding query filter params (role.id=a0524931c0d2413d955919535ffeecaa, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=aab2ee7790444760b8333226be499621) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:00.925286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:00.926817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:00.937563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 500/990] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:00 2022] GET /identity/v3/role_assignments?user.id=aab2ee7790444760b8333226be499621&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=a0524931c0d2413d955919535ffeecaa => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:00.982009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 491/991] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:01.251379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.355276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5f31-71fc-42ba-9226-d470c34b9aa1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:01.392368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:01.393051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.393328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.393595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.394254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:01.395686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-add495ce-4257-4af1-bb34-587afdb2d3fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:01.405526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:01.406577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:01.407455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:01.408852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 501/992] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:01.412509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.460385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:01.493669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02aca5-b5c5-481d-af42-9d6df9054417 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.494112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02aca5-b5c5-481d-af42-9d6df9054417 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.494389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be02aca5-b5c5-481d-af42-9d6df9054417 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.496005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 502/993] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:01.501178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.501464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.501738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.514962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:01.515692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe72d-9011-4b18-aa08-23ee2a498364 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:01.522885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:01.524693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:01.552996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:01.554191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.555081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.555822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.556907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:01.557439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:01.559616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:01.572987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 492/994] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:01.575022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.611715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 503/995] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:01.686350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd895a93-4741-4985-bb57-19d5ce1d5754 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:01.697528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:01.723452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:01.724056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.724337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.724626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.725271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] RBAC: Adding query filter params (type=compute) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:01.725574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:01.726943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:01.736936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 493/996] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:01.737643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.842592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176fa17-b31b-4df9-920e-22579b8f69af None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:01.879135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:01.879804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:01.880028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:01.880277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:01.881001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] RBAC: Adding query filter params (interface=public, service_id=9ad0c979d17c4c64aeda4f0f9f797a96, region_id=RegionOne) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:01.881310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:01.882675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:01.885147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:01.887479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.893562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 504/997] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/endpoints?service_id=9ad0c979d17c4c64aeda4f0f9f797a96&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:01.894427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:01.997241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5ac6c-4ce1-48b0-b8c5-5cc264a3d5ca None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:02.000736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782e0e1-6145-496a-b9a3-32b7f0824f51 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:02.035925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:02.036558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:02.036873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:02.037150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:02.037731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:02.038544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:02.039098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:02.039196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:02.039408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:02.039687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:02.040284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:02.041665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:02.048071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 505/998] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:02.049000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:02.049971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:02.051398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 494/999] 10.222.0.27 () {56 vars in 1126 bytes} [Tue Sep 20 04:40:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:02.062099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:02.137259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:02.182305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ed739-84e8-4bcc-a805-f39ece644894 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:02.219978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:02.220998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:02.220998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:02.221206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:02.222770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:02.222770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:02.224499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:02.229714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 506/1000] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:40:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:02.235671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:02.277748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:02.339636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290c9d3-219a-4443-9341-11457480fb76 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:02.374872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:02.375473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:02.375755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:02.376081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:02.376739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:02.377043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:02.378435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f6565-7b62-459e-8d21-6849223e97e1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:02.383633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 495/1001] 10.222.0.27 () {56 vars in 1122 bytes} [Tue Sep 20 04:40:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:02.389147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:02.493403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7ff44-f222-487d-a80c-e63bc63d0992 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:02.521154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:02.529520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:02.530073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:02.530357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:02.530634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:02.531315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:02.532784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c34ed-1194-4727-b5b8-720cf651398f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:02.551053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 507/1002] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:40:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:03.163350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121df78-570c-48b8-91fb-b4e22672b1cc None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:03.163806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121df78-570c-48b8-91fb-b4e22672b1cc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:03.163886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121df78-570c-48b8-91fb-b4e22672b1cc None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:03.165390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 496/1003] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:03.170350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:03.170737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:03.170905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:03.189300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:03.190607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:03.263425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 508/1004] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:03.535279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121df78-570c-48b8-91fb-b4e22672b1cc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:03.635807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121df78-570c-48b8-91fb-b4e22672b1cc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:03.637438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:03.671252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:03.671880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:03.672161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:03.672434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] PATH_INFO: `/v3/services/image` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:03.673046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:03.674507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:03.683573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:03.684565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:03.685457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:03.686838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 497/1005] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Sep 20 04:40:03 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:03.690556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:03.793020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250eeb0-bc80-4e36-b0ed-c49511c6c9a6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:03.807908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:03.829988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:03.830601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:03.830871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:03.831142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:03.832386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] RBAC: Adding query filter params (name=image) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:03.832727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:03.834093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cd50b-23b3-4061-9484-de695e7b21dd None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:03.839345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 509/1006] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:03.844859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:03.946590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be08d6-49ba-40de-b8c0-638deb47603f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:03.980223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:03.980863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:03.981149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:03.981421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:03.982056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] RBAC: Adding query filter params (type=image) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:03.982361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:03.983724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc6a61-6b52-46d9-b0b5-d7253429b99c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:03.992645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 498/1007] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:04.250493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cd0f-dd47-46d5-968d-6e246cea81b9 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.251033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cd0f-dd47-46d5-968d-6e246cea81b9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.251033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79cd0f-dd47-46d5-968d-6e246cea81b9 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.252499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 510/1008] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:04.258243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c961b5-6974-4040-a1b0-14443c15a08a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.258630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c961b5-6974-4040-a1b0-14443c15a08a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.258807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c961b5-6974-4040-a1b0-14443c15a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.277589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d8c961b5-6974-4040-a1b0-14443c15a08a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:04.278789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d8c961b5-6974-4040-a1b0-14443c15a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:04.302893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:04.347859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 499/1009] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:04.615270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79cd0f-dd47-46d5-968d-6e246cea81b9 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:04.724868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79cd0f-dd47-46d5-968d-6e246cea81b9 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:04.758927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb8f54-37ee-495f-8948-7739d37af687 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.759210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb8f54-37ee-495f-8948-7739d37af687 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.759491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb8f54-37ee-495f-8948-7739d37af687 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.759601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:04.760155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.760431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.760736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.761091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 500/1010] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:04.761316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:04.762672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:04.768078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.768637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.769193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:04.771844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:04.772909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:04.773898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 511/1011] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:04.778886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:04.793674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:04.794871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:04.823541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:04.865375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 501/1012] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:04.889981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd883a79-6bbe-43e1-8fe1-42742c41e1a3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:04.928213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:04.928213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:04.928213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:04.928213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:04.929754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:04.929754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:04.929754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:04.935413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 512/1013] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:40:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:04.940967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:05.033651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:05.044907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc6774-c0bf-4ec2-8387-e50cb11bd101 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:05.078300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:05.078900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:05.079180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:05.079452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:05.080075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:05.080374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:05.082344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:05.091454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 502/1014] 10.222.0.27 () {56 vars in 1112 bytes} [Tue Sep 20 04:40:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:05.092902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:05.139720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:05.209968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cf790-6207-4dfd-ac7e-8a9cb2caa1b2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:05.217211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:05.246539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:05.247205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:05.247493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:05.247764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:05.248423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:05.249803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:05.255011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e31935-3468-4dad-9247-362761ee21b4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:05.280813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 513/1015] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:40:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:40:05.302579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:05.303888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:05.304378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:05.304865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:05.306279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:05.307959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21b535-626d-47b7-990d-e002e3f7d9f0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:05.310239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 503/1016] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:05.315859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:05.427646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e34a2-bfc0-4047-ac75-fd6aca33dd7a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:05.467463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:05.468136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:05.468422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:05.468734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:05.469396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:05.470817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14427b5c-c719-4038-b76a-8a1cf0a1cf7b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:05.498940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 514/1017] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:40:05 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:06.166339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdbb2b-6862-4797-bd5f-05c4f65bdf68 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:06.166632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdbb2b-6862-4797-bd5f-05c4f65bdf68 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:06.166913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdbb2b-6862-4797-bd5f-05c4f65bdf68 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:06.168401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 504/1018] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:06.173607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d1ce-a15f-4c4e-a50b-80b13468bfb0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:06.173896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d1ce-a15f-4c4e-a50b-80b13468bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:06.174170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d1ce-a15f-4c4e-a50b-80b13468bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:06.192571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a041d1ce-a15f-4c4e-a50b-80b13468bfb0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:06.193785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a041d1ce-a15f-4c4e-a50b-80b13468bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:06.203793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:06.265124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 515/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:06 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:06.541043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdbb2b-6862-4797-bd5f-05c4f65bdf68 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:06.561638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:06.648925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdbb2b-6862-4797-bd5f-05c4f65bdf68 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:06.683690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:06.684288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:06.684563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:06.684861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:06.685512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:06.686865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e61d41-5ae0-4ba6-a686-0d251e8c5f92 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:06.703150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 505/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:40:06 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:07.572904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28340e3f-0a8d-4fd6-a095-383c0182ed5e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:07.573427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28340e3f-0a8d-4fd6-a095-383c0182ed5e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:07.573427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28340e3f-0a8d-4fd6-a095-383c0182ed5e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:07.574914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 516/1021] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:07.580038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc709d9-82e3-4545-b902-419784c54c29 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:07.580486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc709d9-82e3-4545-b902-419784c54c29 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:07.580561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc709d9-82e3-4545-b902-419784c54c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:07.601081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6fc709d9-82e3-4545-b902-419784c54c29 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:07.602320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6fc709d9-82e3-4545-b902-419784c54c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:07.688633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 506/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:07.714069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80323446-853b-45c8-9de3-db602b832d28 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:07.714355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80323446-853b-45c8-9de3-db602b832d28 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:07.714627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80323446-853b-45c8-9de3-db602b832d28 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:07.716074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 517/1023] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:07.721367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:07.721639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:07.722428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:07.729644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:07.741900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:07.743083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:07.810221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 507/1024] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:07.958325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80323446-853b-45c8-9de3-db602b832d28 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.060515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80323446-853b-45c8-9de3-db602b832d28 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.076401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.094701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.095307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.095582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.095853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.106579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:08.111360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:08.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:08.113303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 518/1025] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:40:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:08.118221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.122200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.187097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00cc5f-3741-4583-bc8f-a8601740041e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.223895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f023d6d-a6ce-4a84-8144-be2e296b2643 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.224391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.225057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.225330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.225591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.227334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.228785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.230518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 508/1026] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:08.235882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.238964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.263463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.264678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.265166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.265691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.266780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:08.267152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.268516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.282839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 519/1027] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:08.283834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.344792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5e1aa-e197-47dd-81f6-e3952fce8ab9 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.355124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.382135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.382742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.383022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.383295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.383947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.385406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.403441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff54366-8c75-4cdd-a9a3-0bf655bcf46f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.420871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 509/1028] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:40:08 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:40:08.445002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.445658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.445935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.446205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.459246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.460744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:08.477273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:08.478315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:08.479242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:08.479242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:08.479242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:08.479242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 520/1029] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:08.488658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.551789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.599857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db162a2-b732-4b5a-87de-a0458cfb3f96 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.635438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.636048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.636332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.636662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.637282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:08.637584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.638956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.655906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 510/1030] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:08.662477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.768697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60babd6-000e-4e31-9424-7ebed5687d68 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.789296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.809549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.810186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.810467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.810740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.819209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.820746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-463ae597-f362-44db-99d6-8ea9b9d8db7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:40:08.831943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:08.832925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:08.833912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 521/1031] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:08.840317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:08.896305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660924e2-e067-476b-9698-cc037869d6dc None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.896613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660924e2-e067-476b-9698-cc037869d6dc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.896922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660924e2-e067-476b-9698-cc037869d6dc None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.898271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 522/1032] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:08.903632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.903983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.904323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.922406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:08.923610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:08.948461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5e28f-2b06-4388-8fde-85753613f4db None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:08.969064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:08.988231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:08.988791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:08.989074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:08.989348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:08.990072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:08.990375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:08.991813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:08.994222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 523/1033] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:09.018204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 511/1034] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:40:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:09.019656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.124431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0941795-5cfc-4910-9997-37e4f7c612dc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.135722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.165380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.165997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.166273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.166542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.235593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.237155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.269842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.349261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 524/1035] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:09.354969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.367465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.380503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50484ffd-5c93-4839-bfbb-ade23e3c92ad None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.417979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c4be-090d-4824-a213-c045babee960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.418699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c4be-090d-4824-a213-c045babee960 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.418988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c4be-090d-4824-a213-c045babee960 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.419270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c4be-090d-4824-a213-c045babee960 None admin] PATH_INFO: `/v3/services/image` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.419854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c4be-090d-4824-a213-c045babee960 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.421252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c4be-090d-4824-a213-c045babee960 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-4991c4be-090d-4824-a213-c045babee960 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:09.430334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:09.431365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:09.432265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 512/1036] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:09.437601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c4be-090d-4824-a213-c045babee960 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.468423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa52ca-b22f-45b7-8203-6031df6f8e6b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.482665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.506452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.507068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.507341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.507612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.517463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.518931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:09.535072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:09.536011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:09.537520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:09.537520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:09.537520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:09.540779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 525/1037] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:09.541585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.548872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c4be-090d-4824-a213-c045babee960 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.584292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.584929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.585206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.585479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.586102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] RBAC: Adding query filter params (name=image) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:09.586401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.587771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.592833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 513/1038] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:09.598974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.609063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.657237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47d041-3cfc-45ce-aaf7-9c1d962e2847 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.693987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.695044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.695044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.695331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.695949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:09.696252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.697799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.706992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25beb7be-9bcd-494e-bfd1-bd2786ae9b93 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.709790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.716097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 526/1039] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:09.716847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.743218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.744065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.744341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.744641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.745260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:09.745560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.746939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.752635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 514/1040] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:09.757851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.831114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94960a-36b0-4318-85a6-71703315ee1b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.850750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:09.865075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f22a8-58d5-4e9e-8231-62dafefd26c7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:09.866091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.866718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.866991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.867258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.873753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.875176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:40:09.887314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:09.888225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:09.889786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 527/1041] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:09.894141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.903079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:09.903683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:09.903961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:09.904230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:09.905303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] RBAC: Adding query filter params (interface=public, service_id=7d6c15b6840d4689bcf754e8ef68a24c, region_id=RegionOne) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:09.905610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:09.907281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:09.912303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 515/1042] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/endpoints?service_id=7d6c15b6840d4689bcf754e8ef68a24c&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:09.917870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:09.951649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:10.002793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f99fb3-9553-46e5-8fd6-2b6df254bd5d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:10.024782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae08a3-e117-4323-bfda-5fc57715adb6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:10.040554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:10.041194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:10.041474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:10.041743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:10.042424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:10.042726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:10.044145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:10.057889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 528/1043] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:10.064301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:10.065735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:10.066398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:10.066680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:10.066976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:10.067571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:10.068962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285579f9-d48c-4f9d-b9c7-c1bdec846371 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:10.076084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 516/1044] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:40:09 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:40:10.117882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:10.185891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ad439-4920-4a8e-a167-08f6097b47f2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:10.223422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:10.224048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:10.224324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:10.224590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:10.225352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] RBAC: Adding query filter params (role.id=a0524931c0d2413d955919535ffeecaa, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:10.225660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:10.227135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ecbd6-181f-431b-b366-a3e6f925c986 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:10.240934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 529/1045] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:10 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=a0524931c0d2413d955919535ffeecaa => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:10.656164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3dbd-625e-421b-b321-701e0d64b684 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:10.656513 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3dbd-625e-421b-b321-701e0d64b684 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:10.656758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b3dbd-625e-421b-b321-701e0d64b684 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:10.658139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 517/1046] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:10.664198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:10.664491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:10.664798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:10.682795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:10.683976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:10.750794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 530/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:11.024967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b3dbd-625e-421b-b321-701e0d64b684 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:11.123630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:11.136336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b3dbd-625e-421b-b321-701e0d64b684 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:11.176227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:11.177139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:11.177607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:11.178063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:11.188099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:11.193154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:11.194108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:11.195006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:11.196956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 518/1048] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:11.203065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:11.237127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:11.305500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ad2d-2a91-4309-b808-3cc59536964c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:11.340136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:11.340785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:11.341067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:11.341338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:11.341999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:11.342299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:11.343663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:11.356666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 531/1049] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:11.358971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:11.462638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4af2aa-11ec-4b3c-bfac-1c2ab5b0db8c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:11.495808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:11.496433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:11.496747 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:11.497023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] PATH_INFO: `/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:11.510569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] RBAC: Authorizing `identity:get_user(user_id=8c12e87342d14d3cb34e2e7aafaed8ea)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:11.512009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:11.516947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 519/1050] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:11.519971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:11.614519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:11.622002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4020d3-3cc4-4919-bf54-e534f4c8ee3f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:11.659917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:11.660688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:11.660976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:11.661250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:11.729256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:11.730760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:11.824711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:11.836780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 532/1051] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:40:11.839020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:11.855118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:11.942251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500c607-a5d8-4f48-b4ad-de1839492a4f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:11.976215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:11.976872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:11.977155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:11.977427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:11.978527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:11.979963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442786c-0780-4ed9-8013-f28feb762801 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:11.986721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 520/1052] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:11.988185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.087287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff169a5-4b10-4817-8336-2af85f13ccee None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:12.125454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:12.126080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.126356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.126625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.129244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2, user.id=8c12e87342d14d3cb34e2e7aafaed8ea) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:12.129244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:12.133945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe5f99-d44d-452b-b375-b752928b1cbc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:12.144640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 533/1053] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:11 2022] GET /identity/v3/role_assignments?user.id=8c12e87342d14d3cb34e2e7aafaed8ea&scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:12.231916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860437-9816-4488-ae6d-38d5f849f5b0 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.232423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860437-9816-4488-ae6d-38d5f849f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.232423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7860437-9816-4488-ae6d-38d5f849f5b0 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.234055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 521/1054] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:12.239754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d08-3aa7-4499-ba17-ebc8f4762392 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.240032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d08-3aa7-4499-ba17-ebc8f4762392 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.240306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533d08-3aa7-4499-ba17-ebc8f4762392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.260583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ad533d08-3aa7-4499-ba17-ebc8f4762392 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:12.261898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ad533d08-3aa7-4499-ba17-ebc8f4762392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:12.325969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:12.338906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 534/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:12.385865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e629aa-0b14-4122-a008-59f8c3e29597 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.386148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e629aa-0b14-4122-a008-59f8c3e29597 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.386425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e629aa-0b14-4122-a008-59f8c3e29597 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.387770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 522/1056] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:12.393029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.393316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.394211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.412491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:12.413693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:12.480851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 535/1057] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:12.604836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e629aa-0b14-4122-a008-59f8c3e29597 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.686632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:12.707540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e629aa-0b14-4122-a008-59f8c3e29597 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:12.740619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:12.741234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.741508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.741776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] PATH_INFO: `/v3/services/image` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.742342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:12.743694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:12.749819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:12.752219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:12.753751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 20 04:40:12.754854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:12.758296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 523/1058] 10.222.0.27 () {56 vars in 1105 bytes} [Tue Sep 20 04:40:12 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:12.758956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.857019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcb7dc-09bd-49ee-a581-33a206496288 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:12.861506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bfcde-307a-4f1f-9181-cf58f029a176 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:12.880111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:12.894638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:12.895270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.895547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.895817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.899818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:12.900635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:12.901056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:12.901476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:12.902235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] RBAC: Adding query filter params (name=image) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:12.902694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:12.904064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:12.906325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:12.913665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 524/1059] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:12.913820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:12.914888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:12.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 536/1060] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:40:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:12.917872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.917975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:12.931801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:13.025990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236594b-1963-420a-ace1-bb8d53e2fad4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.028788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9060d6-dcd2-42a6-84ab-df35d68174bc None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.060078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785d314-505d-4718-802c-660632937e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.060701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d314-505d-4718-802c-660632937e69 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.060998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d314-505d-4718-802c-660632937e69 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.061255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785d314-505d-4718-802c-660632937e69 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.061878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785d314-505d-4718-802c-660632937e69 None admin] RBAC: Adding query filter params (type=image) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:13.062178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785d314-505d-4718-802c-660632937e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.063530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785d314-505d-4718-802c-660632937e69 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.064861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.066173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.066650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.067098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.067920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:13.068398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.068855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 525/1061] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:13.070180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.074401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785d314-505d-4718-802c-660632937e69 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.084930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 537/1062] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:40:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:13.087664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.133529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:13.155353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:13.184416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785d314-505d-4718-802c-660632937e69 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.195312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9f52-5c24-461d-9fa2-b14d66eae2d6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.217932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.218527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.218804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.219217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.219736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.221137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.230495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.231189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.231468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.231732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.250247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 526/1063] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:40:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:40:13.250777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.250777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:13.266615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:13.267685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:13.268690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:13.268690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:13.268690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:13.268690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 538/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:13.273898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.400314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8bf2d-e3bf-493f-9285-305eb517034a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.430690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:13.442080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.443069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.443069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.443271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.444191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:13.444191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.445681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.457639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 527/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:13.463493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.557256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:13.564685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a8c71-e8a1-4cc7-aa68-e0d307bd47b1 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.598015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.599142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.599419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.599685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.607182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.608628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:40:13.619597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:13.620533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:13.621808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 539/1066] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:13.626657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.736120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb790381-8b2d-4979-9e4d-9d8d13d3ac98 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.771891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.772499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.772820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.773092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.773821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:13.774127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.775804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.797824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 528/1067] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:13.799544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:13.898414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcbe90-603c-489f-aba0-e946845e1fd6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:13.932514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:13.933167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:13.933443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:13.933709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:13.944186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:13.945642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:13.959659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:13.960905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:13.961842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:13.961842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:13.961842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:13.965400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 540/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:13.966854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.033921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:14.098027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a8ebd-662e-4d0c-9be7-8c0b114f5105 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:14.136595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:14.137285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.137565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.137831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.138509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:14.138815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:14.140162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:14.156540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 529/1069] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:14.161818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.172475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:14.269980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d594d57-4a67-4e26-8338-01fd0ad9bdfa None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:14.304725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:14.305326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.305606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.305873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.312433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:14.313898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-ae978d1a-4121-4768-a31f-90e1ef71b296 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:40:14.324896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:14.325827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:14.326753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:14.328682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 541/1070] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:14.332644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.363864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b0cf-61f2-47c0-9500-7298f06181e2 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.364193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b0cf-61f2-47c0-9500-7298f06181e2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.364415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97b0cf-61f2-47c0-9500-7298f06181e2 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.365783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 542/1071] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:14.370746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.371133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.371405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.389962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:14.391143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:14.442096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb9336-27b4-4a0f-9649-d7e3ce87ee41 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:14.444506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:14.460004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:14.472640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 543/1072] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:14.477059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:14.477664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.477944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.478215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.478898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:14.479201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:14.480774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:14.494590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 530/1073] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:14.498527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.596888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d51e1-2c34-49dc-9db1-f1688678436a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:14.630074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:14.630728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.631004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.631278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/a0524931c0d2413d955919535ffeecaa` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.633437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=9809a4423cfb4ddda8d03e663f81b060, role_id=a0524931c0d2413d955919535ffeecaa)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:14.634936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:14.652640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 544/1074] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:40:14 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/9809a4423cfb4ddda8d03e663f81b060/roles/a0524931c0d2413d955919535ffeecaa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:40:14.741998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.851008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaea3e3-1cc8-4352-835b-f069cffb53dd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:14.883708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:14.884335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:14.884628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:14.884907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:14.895745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:14.900922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:14.901910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:14.902791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 531/1075] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:14.910286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:14.966512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:15.013757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583961d-c19e-4350-9924-8a68bbf6e88b None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:15.047494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:15.048107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.048385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.048679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.049333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:15.050163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:15.051521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:15.064474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 545/1076] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:15.066404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:15.091748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:15.174976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225d602-9081-4a82-9935-e27420bafbc5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:15.209091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:15.209717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.209995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.210263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] PATH_INFO: `/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.211368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=8c12e87342d14d3cb34e2e7aafaed8ea)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:15.212814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:15.219245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 532/1077] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:15 2022] GET /identity/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:15.220100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:15.320082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3b334-9bff-4ff7-86c6-2ded7370902f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:15.338469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:15.362870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:15.363532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.363809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.364080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.365326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:15.366812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9b4db-ee2c-4c87-8f1f-a1a03333abe4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:15.368695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 546/1078] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:15 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:15.374641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:15.461393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a048cf-48f0-4f36-b53d-078200fa642c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.461913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a048cf-48f0-4f36-b53d-078200fa642c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.461913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a048cf-48f0-4f36-b53d-078200fa642c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.463363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 547/1079] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:15.468212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.468494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.468805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.472788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:15.486137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9aa7-7be9-42a6-b2ed-ce52b8644aa2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:15.487643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:15.488841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:15.529051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:15.530689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.531175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.531669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2/users/8c12e87342d14d3cb34e2e7aafaed8ea/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:15.534837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] RBAC: Authorizing `identity:create_grant(project_id=f579f2d0fc0e4bfa87ed0aae743127e2, user_id=8c12e87342d14d3cb34e2e7aafaed8ea, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:15.536967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:15.554561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 533/1080] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:40:15 2022] PUT /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2/users/8c12e87342d14d3cb34e2e7aafaed8ea/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:40:15.564654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 548/1081] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:15.839188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:15.944993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9ea97-e04a-4b4f-84e0-39176dbbc8fb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:15.980251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:15.980889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:15.981172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:15.981444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:16.003340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:16.066932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:16.068397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:16.171461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 534/1082] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:16.177999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:16.277863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208971c-a7ed-4cb6-854d-8d3c8b0e9445 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:16.294737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:16.313884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:16.314519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:16.314798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:16.315069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:16.325332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:16.326778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-fd0ce11b-bee9-419c-b984-aaca677559ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:16.341693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:16.342663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:16.343597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:16.343597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:16.343597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:16.344468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 549/1083] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:16.350384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:16.419989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:16.468986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5903-519a-4403-bf8b-98142a212052 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:16.502806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:16.503406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:16.503686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:16.503958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:16.504584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:16.504905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:16.506264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cfdd66-d82d-4ad3-b0c7-a58bf6b9b03f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:16.524634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 535/1084] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:16.880189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970817-2883-4712-88b5-b8822b234350 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:16.880474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970817-2883-4712-88b5-b8822b234350 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:16.880777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970817-2883-4712-88b5-b8822b234350 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:16.882123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 550/1085] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:16.887440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:16.887732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:16.888010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:16.906596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:16.907769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:16.972136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 536/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:17.244332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9970817-2883-4712-88b5-b8822b234350 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:17.348154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9970817-2883-4712-88b5-b8822b234350 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:17.380304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:17.380951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.381230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.381502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.391831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:17.397060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:17.398054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:17.399684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 551/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Tue Sep 20 04:40:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:17.404324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:17.446612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:17.501804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914604c-7254-456c-8e61-6abf8efbf532 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:17.535898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:17.536367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.536587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.536884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.537545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:17.537847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:17.539205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba8585-9a85-425b-8979-c688d8d0cbf7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:17.547160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 537/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:40:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:17.552944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:17.582587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:17.656054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a0e59-bddd-4030-832d-46d9d80d476a None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:17.691514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:17.692140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.692420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.692731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.701225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f45a2-ddbb-4725-9a93-bf1c9edb59cb None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.701455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f45a2-ddbb-4725-9a93-bf1c9edb59cb None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.701727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f45a2-ddbb-4725-9a93-bf1c9edb59cb None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.703066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 538/1089] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:40:17.703395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:17.704867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:17.710350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.710633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.710916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:17.718816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:17.719849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:17.721180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:17.721180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:17.721180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:17.721180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 552/1090] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:17.726269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:17.731818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:17.733055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:17.801612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 539/1091] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:17.806809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:17.825779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91442b22-214d-40dc-9698-800ae3905fac None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:17.857127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:17.857713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:17.857987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:17.858253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:17.858873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:17.859168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:17.860538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:17.872349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 553/1092] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:17.878795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:17.911032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:17.974104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26ba3b-5d59-4edb-a742-c5058151d91e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.006708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.007319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.007595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.007862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.014514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.015926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:40:18.027196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:18.028303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:18.029300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 540/1093] 10.222.0.27 () {56 vars in 1093 bytes} [Tue Sep 20 04:40:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:18.034069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.074049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.131313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b7c03-a7ee-4285-8a98-1918986f7e33 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.167938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.168543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.168852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.169125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.169922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:18.170229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.171645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.171771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.179864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763c629-4410-4651-b42f-a9226906c313 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.181693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.189762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 554/1094] 10.222.0.27 () {56 vars in 1133 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:18.196118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.214536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.215162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.215444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.215714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.226401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:18.232447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:18.233456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:18.234358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:18.236744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 541/1095] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:18.240167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.315602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9364b8-d026-4942-be04-5270d0150e1c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.352338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16f1b-229e-4215-9024-5cf2405b78ab None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.354135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.354819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.355156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.355491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.389433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.390041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.390321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.390598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.391288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:18.391584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.392985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.402020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 542/1096] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:18.407863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.426833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.427740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.432494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.434028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.522016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481039a-f6dd-4ce3-b09b-70166d488a3c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.544992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 555/1097] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:40:18.550815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.560807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.561542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.561933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.562512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] PATH_INFO: `/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.563573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] RBAC: Authorizing `identity:get_user(user_id=8c12e87342d14d3cb34e2e7aafaed8ea)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.565196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.567076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 543/1098] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/users/8c12e87342d14d3cb34e2e7aafaed8ea => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:18.573451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.648911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.653100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d101fb9-a2cb-4a35-8dcc-70fee04f8510 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.657339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.673620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5cdb3-60c3-4cd3-b737-e78f463b7619 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.687436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.688070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.688335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.688677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.698298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.699730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.706762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.707369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.707630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.707897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:18.714358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:18.715505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:40:18.716469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:18.716469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:40:18.716469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:18.716469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 556/1099] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:18.721195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.776354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.777757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.818989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5c28-26dd-4122-89e7-4b6f8596d16f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.852180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:18.852820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:18.853104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:18.853366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:18.853986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:18.854284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:18.855630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:18.858721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.867243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 557/1100] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:18.872857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.881004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 544/1101] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:18.882028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:18.884304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:18.969648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95964d98-1f3e-4a8b-ad88-c289314d37dc None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:18.976743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec86724-9f48-4b0b-8f56-ed95d6c831ae None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:19.003794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:19.004417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:19.004729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:19.005012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:19.010394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:19.011012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:19.011287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:19.011485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:19.011590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:19.012682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:19.012952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:19.014086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:19.015680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 545/1102] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:19.022241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:40:19.024842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:19.025881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:19.026871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 558/1103] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:19.031547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:19.073389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:19.100158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:19.120052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f99b-f4a9-4587-8116-13e16ec013eb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:19.128983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4932b-8476-44f1-a956-a1f88c9908b3 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:19.156657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:19.157267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:19.157548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:19.157823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:19.158545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2, user.id=8c12e87342d14d3cb34e2e7aafaed8ea) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:19.158882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:19.160305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:19.165162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:19.165766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:19.166044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:19.166184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 546/1104] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:19 2022] GET /identity/v3/role_assignments?user.id=8c12e87342d14d3cb34e2e7aafaed8ea&scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:19.166383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:19.166983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:19.167285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:19.168731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52dd7a-4aed-4803-b3f4-aaf3d13903ae None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:19.188994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 559/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:19.190999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:19.295617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61100820-eba1-44d7-ad73-f12676c16b48 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:19.313337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:19.332257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:19.332896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:19.333177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:19.333448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:19.334169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] RBAC: Adding query filter params (role.id=a0524931c0d2413d955919535ffeecaa, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=9809a4423cfb4ddda8d03e663f81b060) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:19.334474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:19.335897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3cac0-657f-4b4f-bc14-c4090fba7da5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:19.342131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 547/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:19 2022] GET /identity/v3/role_assignments?user.id=9809a4423cfb4ddda8d03e663f81b060&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=a0524931c0d2413d955919535ffeecaa => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:21.315185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692d41-7493-456d-8bb6-bf7177d59a4a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:21.315706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692d41-7493-456d-8bb6-bf7177d59a4a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:21.315706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b692d41-7493-456d-8bb6-bf7177d59a4a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:21.317100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 560/1107] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:21.322151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:21.322527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:21.322700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:21.340267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:21.341478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:21.405027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 548/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:21.676414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b692d41-7493-456d-8bb6-bf7177d59a4a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:21.777550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b692d41-7493-456d-8bb6-bf7177d59a4a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:21.804193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:21.810138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:21.810749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:21.811023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:21.811285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:21.812358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:21.813813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a68f1-2079-422c-91a1-9ae49f1f51b4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:21.815361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 561/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:21.821061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:21.915745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b904605-7bdb-444d-9e50-5a94f97151ef None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:21.925980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:21.949001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:21.949531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:21.949807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:21.950073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:21.950718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:21.952134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320ebd0-7082-409b-9d05-2dc4260b4cdb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:21.972424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 549/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:40:21 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:40:24.166137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235637-7382-4845-9083-3344c74476b2 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:24.166565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235637-7382-4845-9083-3344c74476b2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:24.166688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235637-7382-4845-9083-3344c74476b2 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:24.168000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 562/1111] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:24.173282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:24.173786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:24.173878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:24.192834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:24.194133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:24.259243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 550/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:24.550115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235637-7382-4845-9083-3344c74476b2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:24.652057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235637-7382-4845-9083-3344c74476b2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:24.683930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:24.684540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:24.684840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:24.685108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] PATH_INFO: `/v3/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:24.686448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] RBAC: Authorizing `identity:get_role(role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:24.687806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:24.689143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 563/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:24 2022] GET /identity/v3/roles/d53450bb086544ce968b603c0997c5b9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:24.694974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:24.796080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:24.802868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be48dc-b957-47e6-a4c4-edfdbed52857 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:24.835547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:24.836169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:24.836448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:24.836748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] PATH_INFO: `/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:24.849793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] RBAC: Authorizing `identity:get_user(user_id=3c1a979ecb4d4c83b17147c0ef140d7c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:24.851233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:24.852850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 551/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:24 2022] GET /identity/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:24.858643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:24.922476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:24.957166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29017cf-46c0-41f1-8dd9-916fa35ed17e None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:24.989274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:24.989868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:24.990145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:24.990414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:25.054707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:25.056078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:25.115810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:25.151931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 564/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:25.157614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:25.251387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ccec-ae3c-49d0-a281-4276647a435a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:25.284488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:25.285121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:25.285400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:25.285670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:25.286756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:25.288176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc8110-074c-4a75-8520-c174403ad700 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:25.289788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 552/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:25 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:25.295752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:25.390603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f751-6394-4ad2-a9b1-89a82f087025 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:25.422445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:25.423030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:25.423305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:25.423571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:25.424816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2, user.id=3c1a979ecb4d4c83b17147c0ef140d7c) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:25.425118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:25.426511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823965f2-74ce-4986-b1a3-636d6cf91809 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:25.431608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 565/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:25 2022] GET /identity/v3/role_assignments?user.id=3c1a979ecb4d4c83b17147c0ef140d7c&scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2&role.id=d53450bb086544ce968b603c0997c5b9 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:27.520682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ed60-de4e-4957-bcd7-ea7fe59fe431 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:27.521215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ed60-de4e-4957-bcd7-ea7fe59fe431 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:27.521215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ed60-de4e-4957-bcd7-ea7fe59fe431 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:27.522494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 553/1118] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:27.527938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:27.528324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:27.528490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:27.546702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:27.547888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:27.570973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:27.613704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 566/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:27.885644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ed60-de4e-4957-bcd7-ea7fe59fe431 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:27.910899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:27.985126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ed60-de4e-4957-bcd7-ea7fe59fe431 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:28.018279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:28.018899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:28.019178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:28.019445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] PATH_INFO: `/v3/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:28.020829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] RBAC: Authorizing `identity:get_role(role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:28.022265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:28.023563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 554/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:27 2022] GET /identity/v3/roles/d53450bb086544ce968b603c0997c5b9 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:28.029604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:28.122777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f6df9-1f23-42ba-812b-b727c034f43c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:28.156795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:28.157420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:28.157692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:28.157956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] PATH_INFO: `/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:28.159060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=3c1a979ecb4d4c83b17147c0ef140d7c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:28.160467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:28.162120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 567/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:28 2022] GET /identity/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:28.167922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:28.254342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:28.264186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b9d45-d142-4e68-8ef2-3b2f09a68247 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:28.296454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:28.297089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:28.297367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:28.297636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:28.298724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:28.300675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4b6af-744e-4886-bbd6-92b0730f7ed4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:28.302251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 555/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:28 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:40:28.308069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:28.335280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:28.403933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaca007-b232-44d0-b35d-d72cadcfc4b4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:28.435859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:28.436431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:28.436733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:28.437002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2/users/3c1a979ecb4d4c83b17147c0ef140d7c/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:28.439062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f579f2d0fc0e4bfa87ed0aae743127e2, user_id=3c1a979ecb4d4c83b17147c0ef140d7c, role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:28.441083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760c709-7126-4f11-bbec-e4948ed230b0 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:28.452996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 568/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:40:28 2022] PUT /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2/users/3c1a979ecb4d4c83b17147c0ef140d7c/roles/d53450bb086544ce968b603c0997c5b9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:40:30.551667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0452-889a-4e4e-a749-4028c491911c None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:30.552240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0452-889a-4e4e-a749-4028c491911c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:30.552240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e0452-889a-4e4e-a749-4028c491911c None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:30.553560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 556/1124] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:30.558663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:30.559114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:30.559226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:30.576930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:30.578099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:30.641528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 569/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:30.906176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0452-889a-4e4e-a749-4028c491911c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:31.023542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e0452-889a-4e4e-a749-4028c491911c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:31.055396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:31.056008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:31.056286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:31.056557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] PATH_INFO: `/v3/roles/d53450bb086544ce968b603c0997c5b9` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:31.057919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] RBAC: Authorizing `identity:get_role(role_id=d53450bb086544ce968b603c0997c5b9)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:31.059279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:31.060571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 557/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:30 2022] GET /identity/v3/roles/d53450bb086544ce968b603c0997c5b9 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:31.066914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:31.085209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:31.164363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e1ffd-211f-4929-84d6-49e95126c0de None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:31.197437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:31.198054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:31.198332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:31.198597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] PATH_INFO: `/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:31.199686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] RBAC: Authorizing `identity:get_user(user_id=3c1a979ecb4d4c83b17147c0ef140d7c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:31.201130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:31.202679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 570/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:31 2022] GET /identity/v3/users/3c1a979ecb4d4c83b17147c0ef140d7c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:31.209030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:31.236276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:31.307536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca3379-0cef-4f01-b9ae-b076ed8ea53c None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:31.340129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:31.340755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:31.341035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:31.341304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:31.405020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:31.406460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:31.431289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:31.500133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 558/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:31.506771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:31.587778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:31.602304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f0e53-a681-43ab-a9e0-67694cd1be51 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:31.638132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:31.638759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:31.639037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:31.639308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] PATH_INFO: `/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:31.640396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] RBAC: Authorizing `identity:get_project(project_id=f579f2d0fc0e4bfa87ed0aae743127e2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:31.641860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236e020-93fc-4128-bbd7-edb8a5f2f0cc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:31.643509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 571/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:31 2022] GET /identity/v3/projects/f579f2d0fc0e4bfa87ed0aae743127e2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:31.649913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:31.749332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a218a8c-dcf1-4b2f-b996-58f4d86c123b None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:31.773535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:31.782879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:31.784039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:31.784316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:31.784590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:31.785332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] RBAC: Adding query filter params (role.id=d53450bb086544ce968b603c0997c5b9, scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2, user.id=3c1a979ecb4d4c83b17147c0ef140d7c) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:31.785638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:31.787057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a4e23-3ba2-4b15-a084-10e8fe5a61b4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:31.792638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 559/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:31 2022] GET /identity/v3/role_assignments?user.id=3c1a979ecb4d4c83b17147c0ef140d7c&scope.project.id=f579f2d0fc0e4bfa87ed0aae743127e2&role.id=d53450bb086544ce968b603c0997c5b9 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:33.968926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deb734-5a58-4471-947e-f485e793dd3f None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:33.969484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deb734-5a58-4471-947e-f485e793dd3f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:33.969484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26deb734-5a58-4471-947e-f485e793dd3f None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:33.970847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 572/1131] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:33.976130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:33.976510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:33.976701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:33.994358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:33.995519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:34.061292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 560/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:34.326091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26deb734-5a58-4471-947e-f485e793dd3f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:34.422651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26deb734-5a58-4471-947e-f485e793dd3f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:34.455476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:34.456094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:34.456374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:34.456671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:34.457763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:34.459177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616ca9-991f-4024-ad23-5bc977f1f861 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:34.460727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 573/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:34.466323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:34.559875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02598c-2e5c-44f6-b190-5985a5bb3176 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:34.579251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:34.592865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:34.593456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:34.593733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:34.594005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:34.594646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:34.596096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e088b-27ad-454a-88ab-6677525c38b1 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:34.621168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 561/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:40:34 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:36.781151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e29b-5595-4ed8-b5a3-b224afe6e283 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:36.781722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e29b-5595-4ed8-b5a3-b224afe6e283 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:36.781722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558e29b-5595-4ed8-b5a3-b224afe6e283 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:36.783581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 574/1135] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:36.788754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:36.789181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:36.789302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:36.806948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:36.808112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:36.871614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 562/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:37.141399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558e29b-5595-4ed8-b5a3-b224afe6e283 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:37.165754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:37.240118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558e29b-5595-4ed8-b5a3-b224afe6e283 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:37.273162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:37.273779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:37.274304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:37.274579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:37.275711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:37.277190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43a3c2-2392-4f80-b621-f304ac8eaf64 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:37.278759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 575/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:37.284455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:37.378800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5764b18-a56a-4f62-b323-036c908c8ade None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:37.380251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:37.412666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:37.413198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:37.413473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:37.413741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:37.414374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:37.415777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9658f1c-88aa-4489-ab4f-26b50ef81082 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:37.436661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 563/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:40:37 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:39.608437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001ecd-7287-4826-8f0f-7df5fec4025d None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:39.609025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001ecd-7287-4826-8f0f-7df5fec4025d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:39.609025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a001ecd-7287-4826-8f0f-7df5fec4025d None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:39.610336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 576/1139] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:39.615558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:39.615974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:39.616106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:39.633612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:39.634784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:39.710554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 564/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:39.992662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a001ecd-7287-4826-8f0f-7df5fec4025d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.080700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:40.090724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a001ecd-7287-4826-8f0f-7df5fec4025d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:40.123288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:40.123894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:40.124167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:40.124432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:40.134844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:40.139505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:40.140690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:40.142018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 577/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:40.147246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.230143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:40.242509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac90118-a2f9-4b02-ad33-4f9853f712eb None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:40.275933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:40.276477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:40.276788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:40.277061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:40.277717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:40.278018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:40.279375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:40.287125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 565/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:40.293037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.390820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b612-93d3-4f72-adf7-3fe12b07654f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:40.427118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:40.427740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:40.428019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:40.428289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] PATH_INFO: `/v3/users/f97057a969ed418e897ec4ba1539200e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:40.442200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] RBAC: Authorizing `identity:get_user(user_id=f97057a969ed418e897ec4ba1539200e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:40.443620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:40.445223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 578/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:40 2022] GET /identity/v3/users/f97057a969ed418e897ec4ba1539200e => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:40.451109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.545165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babd050-83ab-4295-9add-64a220723f73 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:40.577280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:40.577859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:40.578137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:40.578407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:40.595592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:40.642845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:40.644264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:40.737278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 566/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:40.743027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.753931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:40.838622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61551e1f-1e35-4dab-807f-42164b99a53d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:40.869861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:40.870467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:40.870744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:40.871013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] PATH_INFO: `/v3/projects/c8be5b398c0c4cf89013f52b8fca218c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:40.872094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] RBAC: Authorizing `identity:get_project(project_id=c8be5b398c0c4cf89013f52b8fca218c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:40.873530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8e4b-3964-41ee-88d7-9559af0e260a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:40.875101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 579/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:40 2022] GET /identity/v3/projects/c8be5b398c0c4cf89013f52b8fca218c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:40.881518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:40.929229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:40.977281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef7b3-ca39-4f8f-a569-35cab99ed4a0 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:41.008866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:41.009446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:41.009721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:41.009985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:41.010691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=c8be5b398c0c4cf89013f52b8fca218c, user.id=f97057a969ed418e897ec4ba1539200e) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:41.010991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:41.012399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bf09b-4de3-4892-8842-e14f4ac7ffb3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:41.017552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 567/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:40 2022] GET /identity/v3/role_assignments?user.id=f97057a969ed418e897ec4ba1539200e&scope.project.id=c8be5b398c0c4cf89013f52b8fca218c&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:43.093560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612720-3957-468d-aa19-5a20bcf217ab None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:43.094153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612720-3957-468d-aa19-5a20bcf217ab None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:43.094153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02612720-3957-468d-aa19-5a20bcf217ab None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:43.095445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 580/1147] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:43.100678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:43.101093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:43.101229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:43.118952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:43.120126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:43.184915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 568/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:43.456191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02612720-3957-468d-aa19-5a20bcf217ab None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:43.534540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:43.558106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02612720-3957-468d-aa19-5a20bcf217ab None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:43.590752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:43.591372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:43.591651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:43.591924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:43.601631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:43.606381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:43.607510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:43.608623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 581/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:43.614189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:43.657471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:43.708871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f94d69-59ca-49dc-a1fb-0efc19b6b0d7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:43.744336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:43.744955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:43.745230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:43.745496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:43.746147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:43.746448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:43.747792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:43.755988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 569/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:43.761796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:43.857252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ead95c-373b-4535-8fb2-132b71be6d50 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:43.889661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:43.890268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:43.890546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:43.890818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] PATH_INFO: `/v3/users/f97057a969ed418e897ec4ba1539200e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:43.891930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] RBAC: Authorizing `identity:get_user(user_id=f97057a969ed418e897ec4ba1539200e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:43.893402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:43.894933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 582/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:43 2022] GET /identity/v3/users/f97057a969ed418e897ec4ba1539200e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:43.900742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:43.996307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3946-90d1-41fe-b08d-06a9e68ceee2 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:44.002953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:44.028968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:44.029571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:44.029846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:44.030117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] PATH_INFO: `/v3/projects/c8be5b398c0c4cf89013f52b8fca218c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:44.031191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] RBAC: Authorizing `identity:get_project(project_id=c8be5b398c0c4cf89013f52b8fca218c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:44.033237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361bafa4-4085-4c3e-a9a7-492a66944393 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:44.034812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 570/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:43 2022] GET /identity/v3/projects/c8be5b398c0c4cf89013f52b8fca218c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:44.040758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:44.044391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:44.136683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf13d4-5f87-466b-b299-d345ee752f0f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:44.170363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:44.171011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:44.171287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:44.171556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] PATH_INFO: `/v3/projects/c8be5b398c0c4cf89013f52b8fca218c/users/f97057a969ed418e897ec4ba1539200e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:44.173459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8be5b398c0c4cf89013f52b8fca218c, user_id=f97057a969ed418e897ec4ba1539200e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:44.174943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609b8c4-29d0-4b9a-b8c0-77243d49ec99 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:44.187533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 583/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:40:44 2022] PUT /identity/v3/projects/c8be5b398c0c4cf89013f52b8fca218c/users/f97057a969ed418e897ec4ba1539200e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:40:46.337520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1edf7-9586-4b4c-9dd9-dd4bd457fa43 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:46.338082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1edf7-9586-4b4c-9dd9-dd4bd457fa43 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:46.338082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1edf7-9586-4b4c-9dd9-dd4bd457fa43 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:46.339406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 571/1154] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:46.344762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:46.345163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:46.345315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:46.363481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:46.364689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:46.421651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:46.430812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 584/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:46.723428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1edf7-9586-4b4c-9dd9-dd4bd457fa43 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:46.820977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1edf7-9586-4b4c-9dd9-dd4bd457fa43 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:46.853241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:46.853845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:46.854122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:46.854394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:46.864625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:40:46.869346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:46.870450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:40:46.871544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 572/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:40:46.876345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:46.971037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff0c78-bd18-4098-a6a5-a350b5725fa3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:47.008392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:47.009029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:47.009306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:47.009577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:47.010245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:47.010544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:47.011910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:47.021071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 585/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:47.028013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:47.061709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:47.125729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60260-d854-42c0-bb5c-6cec467af129 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:47.160401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:47.161050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:47.161328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:47.161598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] PATH_INFO: `/v3/users/f97057a969ed418e897ec4ba1539200e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:47.162696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] RBAC: Authorizing `identity:get_user(user_id=f97057a969ed418e897ec4ba1539200e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:47.164122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:47.165716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 573/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:40:47 2022] GET /identity/v3/users/f97057a969ed418e897ec4ba1539200e => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:47.171451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:47.226540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:47.268108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac2258-f640-4c42-87ea-085522fb182f None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:47.300132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:47.300747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:47.301023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:47.301287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:47.371399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:47.372866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:47.425765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:47.468807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 586/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:47.475350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:47.571696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319e926-5204-4601-881c-1ad53a7ceac8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:47.573142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:47.605451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:47.606067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:47.606352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:47.606623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] PATH_INFO: `/v3/projects/c8be5b398c0c4cf89013f52b8fca218c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:47.607718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] RBAC: Authorizing `identity:get_project(project_id=c8be5b398c0c4cf89013f52b8fca218c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:47.609181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8289bf-0cc5-42c3-a5aa-8ee2f71026d7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:47.611356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 574/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:40:47 2022] GET /identity/v3/projects/c8be5b398c0c4cf89013f52b8fca218c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:47.617302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:47.714866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2af837-4f50-4160-b31f-0f18f90b68bc None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:47.746452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:47.747039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:47.747316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:47.747584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:47.748287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=c8be5b398c0c4cf89013f52b8fca218c, user.id=f97057a969ed418e897ec4ba1539200e) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:47.748590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:47.750026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00618a31-59d2-4200-806a-2685c9b8f87d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:47.759099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 587/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:40:47 2022] GET /identity/v3/role_assignments?user.id=f97057a969ed418e897ec4ba1539200e&scope.project.id=c8be5b398c0c4cf89013f52b8fca218c&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:49.902576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e0db-8652-47ca-9f4b-2fb3500d68a6 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:49.903114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e0db-8652-47ca-9f4b-2fb3500d68a6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:49.903114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e0db-8652-47ca-9f4b-2fb3500d68a6 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:49.904434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 575/1162] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:49.910224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:49.910611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:49.910772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:49.929024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:49.930199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:49.943589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:49.994688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 588/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:50.258900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975e0db-8652-47ca-9f4b-2fb3500d68a6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:50.356380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975e0db-8652-47ca-9f4b-2fb3500d68a6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:50.387725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:50.388309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:50.388588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:50.388887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:50.427477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:50.456256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:50.457729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:50.550196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 576/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:40:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:40:50.556159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:50.650381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018451-f38d-4b3e-988f-2a5099693974 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:50.683411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:50.683970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:50.684247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:50.684515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:50.691440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:50.692906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-c6c63fe4-1f71-4720-a96a-8b3f9e47a6bb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:40:50.703951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:40:50.705769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:40:50.706850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 589/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:40:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:40:50.711500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:50.777478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:50.808888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c61ec-53a2-462a-84d1-4efc67c940a7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:50.841427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:50.842012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:50.842290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:50.842557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:50.843182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:40:50.843477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:50.844853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f307f2-613a-4f7e-9c55-cc239853f4d0 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:50.851605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 577/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Tue Sep 20 04:40:50 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:52.940893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d185f1-288c-4537-a53f-ab70d2096c21 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:52.941427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d185f1-288c-4537-a53f-ab70d2096c21 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:52.941427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d185f1-288c-4537-a53f-ab70d2096c21 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:52.942761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 590/1167] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:52.948050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02838b-98a4-4855-99fb-a620bdc34182 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:52.948453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02838b-98a4-4855-99fb-a620bdc34182 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:52.948589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02838b-98a4-4855-99fb-a620bdc34182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:52.966970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9e02838b-98a4-4855-99fb-a620bdc34182 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:52.968139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9e02838b-98a4-4855-99fb-a620bdc34182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:53.031775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 578/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:40:53.299881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d185f1-288c-4537-a53f-ab70d2096c21 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:53.312028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:53.400098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d185f1-288c-4537-a53f-ab70d2096c21 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:53.432991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:53.433585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:53.433861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:53.434130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:53.434798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:53.436174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cb541-4c24-4ecb-82e3-ed2e54bae7a7 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:53.452449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 591/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Tue Sep 20 04:40:53 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:40:55.621613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e26e58-4d17-443b-8032-2fa84ff9569a None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:55.622210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e26e58-4d17-443b-8032-2fa84ff9569a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:55.622210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e26e58-4d17-443b-8032-2fa84ff9569a None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:55.623474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 579/1170] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:55.629346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:55.629745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:55.629898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:55.647674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:55.649149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:55.710928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:55.716013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 592/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:55.989036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e26e58-4d17-443b-8032-2fa84ff9569a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:56.054159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:56.091255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e26e58-4d17-443b-8032-2fa84ff9569a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:56.123737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:56.124349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:56.124648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:56.124925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] PATH_INFO: `/v3/domains/e52b47795c374680b9fabf1c9ab39c81` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:56.135345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=e52b47795c374680b9fabf1c9ab39c81)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:56.136842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e1703-cb18-4601-87d7-f8e3e029f68f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:56.138441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 580/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Sep 20 04:40:55 2022] GET /identity/v3/domains/e52b47795c374680b9fabf1c9ab39c81 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:56.144726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:56.239958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e9277-70b1-4fbe-9abb-ee62dd9f1a31 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:56.272351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:56.272978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:56.273258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:56.273527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:56.274435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:56.275845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31505810-209b-480c-903e-dfd0018a1235 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:56.301205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 593/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:40:56 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:40:58.441785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5630c-5cd1-4751-8603-6528d220f6b5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:58.442301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5630c-5cd1-4751-8603-6528d220f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:58.442301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5630c-5cd1-4751-8603-6528d220f6b5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:58.443565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 581/1174] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:40:58.448860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:58.449262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:58.449403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:58.467553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:40:58.468889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:40:58.512937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:58.533658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 594/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:40:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:40:58.798152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5630c-5cd1-4751-8603-6528d220f6b5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:58.895727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5630c-5cd1-4751-8603-6528d220f6b5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:58.917879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:40:58.928821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:58.929430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:58.929707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:58.929975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] PATH_INFO: `/v3/domains/e52b47795c374680b9fabf1c9ab39c81` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:58.931064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] RBAC: Authorizing `identity:get_domain(domain_id=e52b47795c374680b9fabf1c9ab39c81)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:58.932473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219eaa-1c1f-42a3-9926-f6d3770ceefc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:58.934016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 582/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Tue Sep 20 04:40:58 2022] GET /identity/v3/domains/e52b47795c374680b9fabf1c9ab39c81 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:40:58.940235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:40:59.035452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade923f-bbbb-44a7-ae61-02b611b622bb None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:40:59.068037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:40:59.068573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:40:59.068880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:40:59.069150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:40:59.069796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:40:59.071216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd7271-06de-4cb6-a5de-404be3fba422 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:40:59.091933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 595/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:40:58 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:41:01.257477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792d0c-cc39-4265-963a-c997b3670c2b None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:01.258020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792d0c-cc39-4265-963a-c997b3670c2b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:01.258020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792d0c-cc39-4265-963a-c997b3670c2b None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:01.259335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 583/1178] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:41:01.264608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:01.265029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:01.265171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:01.284558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:41:01.285767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:41:01.314279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:01.349774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 596/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:41:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:41:01.615711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7792d0c-cc39-4265-963a-c997b3670c2b None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:01.714783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7792d0c-cc39-4265-963a-c997b3670c2b None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:01.749745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:01.750453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:01.750735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:01.751008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:01.760754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:41:01.765442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:41:01.766612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:41:01.767538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 584/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:41:01.772850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:01.869713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190f133-8ff8-4a99-b7f1-aa7769974215 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:01.901896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:01.902491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:01.902770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:01.903041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:01.903693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:41:01.903994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:01.905387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:01.913623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 597/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:01.919549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:01.971123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:02.017268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03e33-0c81-44fc-b8a5-2f431bdfd3af None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:02.050928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:02.051539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:02.051822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:02.052095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] PATH_INFO: `/v3/users/ab9e93479724413da7a94540c06a7d33` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:02.066481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] RBAC: Authorizing `identity:get_user(user_id=ab9e93479724413da7a94540c06a7d33)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:02.067944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:02.069651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 585/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:41:01 2022] GET /identity/v3/users/ab9e93479724413da7a94540c06a7d33 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:02.075446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:02.113475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:02.182075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cbbde-7289-461a-957b-b62af8aacffe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:02.213944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:02.214536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:02.214811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:02.215076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:02.278820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:02.280247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:02.307720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:02.378942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 598/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:41:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:41:02.384925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:02.481929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a5755-6ff9-4e2e-ae4e-6df5ad89d4af None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:02.492006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:02.514913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:02.515525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:02.515800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:02.516698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] PATH_INFO: `/v3/projects/8c265d49074141ea8bb66c844c8d7b46` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:02.517791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] RBAC: Authorizing `identity:get_project(project_id=8c265d49074141ea8bb66c844c8d7b46)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:02.519200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde9025-a673-4785-9928-f526571387c6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:02.520815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 586/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:41:02 2022] GET /identity/v3/projects/8c265d49074141ea8bb66c844c8d7b46 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:02.527294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:02.621268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e976e5-3681-446c-87e1-88b2e545fe08 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:02.649406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:02.655297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:02.655886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:02.656164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:02.656435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:02.657191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=8c265d49074141ea8bb66c844c8d7b46, user.id=ab9e93479724413da7a94540c06a7d33) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:41:02.657498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:02.658924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f93ce-e588-432a-83ac-1a166f35a8cc None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:02.664097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 599/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:41:02 2022] GET /identity/v3/role_assignments?user.id=ab9e93479724413da7a94540c06a7d33&scope.project.id=8c265d49074141ea8bb66c844c8d7b46&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:04.826033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01856-2f18-4190-b365-f80e872296f2 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:04.826571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01856-2f18-4190-b365-f80e872296f2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:04.826571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01856-2f18-4190-b365-f80e872296f2 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:04.827895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 587/1186] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:41:04.833583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:04.834000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:04.834115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:04.851863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:41:04.853074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:41:04.917401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 600/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:41:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:41:05.183480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01856-2f18-4190-b365-f80e872296f2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:05.279933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01856-2f18-4190-b365-f80e872296f2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:05.312248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:05.313490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:05.313765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:05.314035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:05.323676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:41:05.328422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:41:05.329570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:41:05.330652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 588/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:41:05.335501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:05.433106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809741b7-a5ee-4b1f-a5d2-83c5fbec9160 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:05.455736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:05.466297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:05.466887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:05.467160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:05.467425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:05.468067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:41:05.468362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:05.470228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:05.477926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 601/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:05.483623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:05.522702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:05.580244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9685ec-0f2b-46de-9ad2-fbc26a301dfd None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:05.613086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:05.613696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:05.613976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:05.614248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] PATH_INFO: `/v3/users/ab9e93479724413da7a94540c06a7d33` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:05.615348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] RBAC: Authorizing `identity:get_user(user_id=ab9e93479724413da7a94540c06a7d33)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:05.616809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:05.618350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 589/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:41:05 2022] GET /identity/v3/users/ab9e93479724413da7a94540c06a7d33 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:05.624059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:05.718427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1c078-cbbe-4d18-9fc9-8e5020a73158 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:05.750657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:05.751262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:05.751537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:05.751802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] PATH_INFO: `/v3/projects/8c265d49074141ea8bb66c844c8d7b46` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:05.752907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] RBAC: Authorizing `identity:get_project(project_id=8c265d49074141ea8bb66c844c8d7b46)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:05.754314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c8aa-bb40-433b-8982-3ffffb19d44c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:05.755883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 602/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:41:05 2022] GET /identity/v3/projects/8c265d49074141ea8bb66c844c8d7b46 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:05.761736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:05.859182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0762f4-c43b-4ef6-a31f-35365018bfcc None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:05.866009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:05.892565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:05.893233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:05.893512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:05.893781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] PATH_INFO: `/v3/projects/8c265d49074141ea8bb66c844c8d7b46/users/ab9e93479724413da7a94540c06a7d33/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:05.895636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c265d49074141ea8bb66c844c8d7b46, user_id=ab9e93479724413da7a94540c06a7d33, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:05.897097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6462ffa-f085-4090-a61d-ba479e4e7dc4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:05.909653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 590/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:41:05 2022] PUT /identity/v3/projects/8c265d49074141ea8bb66c844c8d7b46/users/ab9e93479724413da7a94540c06a7d33/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:41:08.004540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cc8f-c5af-4514-9d85-1826dcb7edee None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.005098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cc8f-c5af-4514-9d85-1826dcb7edee None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.005098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cc8f-c5af-4514-9d85-1826dcb7edee None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.006454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 603/1193] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:41:08.011573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.011957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.012126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.030010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:41:08.031184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:41:08.096528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 591/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:41:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:41:08.365787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cc8f-c5af-4514-9d85-1826dcb7edee None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:08.420468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:08.464342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cc8f-c5af-4514-9d85-1826dcb7edee None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:08.495431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:08.496033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.496305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.496571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.507104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:41:08.512156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:41:08.513408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:41:08.514475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 604/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:41:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:41:08.519922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:08.612009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:08.618684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150f1b1-62e2-4575-b886-c1064edda900 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:08.652291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:08.652923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.653204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.653476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.654136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:41:08.654439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:08.655812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:08.663591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 592/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:41:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:08.669484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:08.764349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83992d-cd7b-4155-8352-49cef25e0683 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:08.777954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:08.799192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:08.808384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.812807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.813078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] PATH_INFO: `/v3/users/ab9e93479724413da7a94540c06a7d33` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.817373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] RBAC: Authorizing `identity:get_user(user_id=ab9e93479724413da7a94540c06a7d33)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:08.818799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:08.820368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 605/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Tue Sep 20 04:41:08 2022] GET /identity/v3/users/ab9e93479724413da7a94540c06a7d33 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:08.826221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:08.921267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52261cda-9206-4ef1-80f1-0b77d7273d5a None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:08.953434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:08.954021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:08.954303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:08.954571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:08.979262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:09.018247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:09.019695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:09.113548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 593/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:41:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:41:09.119448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:09.222490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926879e4-7129-4624-8c75-a77d4ebc2253 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:09.254063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:09.254676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:09.254949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:09.255215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] PATH_INFO: `/v3/projects/8c265d49074141ea8bb66c844c8d7b46` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:09.256286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] RBAC: Authorizing `identity:get_project(project_id=8c265d49074141ea8bb66c844c8d7b46)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:09.257730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d293698-2824-4b73-a738-2d12a851967a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:09.259318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 606/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Tue Sep 20 04:41:09 2022] GET /identity/v3/projects/8c265d49074141ea8bb66c844c8d7b46 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:09.265428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:41:09.321462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:09.361960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d70c0-781c-4a8c-a7e8-22d3b20e95d5 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:41:09.395260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:41:09.395860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:41:09.396143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:41:09.396414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:41:09.397159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=8c265d49074141ea8bb66c844c8d7b46, user.id=ab9e93479724413da7a94540c06a7d33) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:41:09.397466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:41:09.399127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0768a-1d76-4660-917e-161a46f6feef None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:41:09.404686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 594/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:41:09 2022] GET /identity/v3/role_assignments?user.id=ab9e93479724413da7a94540c06a7d33&scope.project.id=8c265d49074141ea8bb66c844c8d7b46&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:41:19.250036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=75 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:41:19.402143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=39 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:04.642000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158722-b7af-4124-86d2-aab957d5c824 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:04.643066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158722-b7af-4124-86d2-aab957d5c824 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:04.643066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2158722-b7af-4124-86d2-aab957d5c824 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:04.643911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 607/1201] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:04.649816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915678-c881-472c-8c59-20edf05917e5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:04.650352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915678-c881-472c-8c59-20edf05917e5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:04.650352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00915678-c881-472c-8c59-20edf05917e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:04.671979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-00915678-c881-472c-8c59-20edf05917e5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:04.673450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-00915678-c881-472c-8c59-20edf05917e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:04.753883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 595/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:05.030498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2158722-b7af-4124-86d2-aab957d5c824 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:05.172961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2158722-b7af-4124-86d2-aab957d5c824 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:05.225616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:05.226515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:05.226879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:05.227151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:05.240085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:05.241641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:05.258848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:05.260005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:05.267357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:05.267357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:05.267357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:05.267357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 608/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:05.269520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915678-c881-472c-8c59-20edf05917e5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:05.399164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00915678-c881-472c-8c59-20edf05917e5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:05.431476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:05.440282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:05.440976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:05.441271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:05.441532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:05.442187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:05.442491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:05.443856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2ff1b-5b1b-44d4-9f19-27265bbdce89 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:05.458333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 596/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:05.464263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:05.538450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:05.597338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee763753-91d5-4d58-b304-eaa102ec5793 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:05.638931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:05.639632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:05.639929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:05.640185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:05.640913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:05.642552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea76ebd-2f55-42c5-9a59-5a729e235a3f None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:05.670457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 609/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Tue Sep 20 04:42:05 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:42:08.019641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675ef1-e68c-4276-9cf0-a284eee698a0 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:08.020204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675ef1-e68c-4276-9cf0-a284eee698a0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:08.020204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c675ef1-e68c-4276-9cf0-a284eee698a0 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:08.021427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 597/1206] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:08.027704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:08.028945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:08.028945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:08.047933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:08.049144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:08.125235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 610/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:08.405995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675ef1-e68c-4276-9cf0-a284eee698a0 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:08.521945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c675ef1-e68c-4276-9cf0-a284eee698a0 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:08.562916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:08.563594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:08.563874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:08.564183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:08.577262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:08.582082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:08.583148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:08.584797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 598/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:42:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:42:08.589835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:08.633163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:08.696860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35d825-67e8-4487-9507-ce7d4882d168 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:08.735361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:08.736018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:08.736285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:08.736741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:08.737282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:08.737574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:08.739011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:08.748629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 611/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:42:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:08.754427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:08.812671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:08.871339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffc253-4e65-4fa3-bb78-cd8d1cc9d888 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:08.909867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:08.910500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:08.910781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:08.911091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:08.923115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:08.925574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:08.943625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:08.944860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:08.945788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:08.945788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:08.945788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:08.945788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 599/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:08.950782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:09.069155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:09.072335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f8e2-9325-4bcd-8e00-739a095142b9 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:09.111298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:09.111988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:09.112343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:09.112756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:09.113519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:09.113920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:09.115481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:09.130383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 612/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:09.134889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:09.245115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b17-d6ad-40e4-9ad1-c117b42de193 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:09.246887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:09.282820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:09.283545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:09.283893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:09.284227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:09.291813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:09.293353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:09.305278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:09.308020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:09.309138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 600/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:09.312413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:09.430096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974cc27-8d71-44fa-9d02-02c7717308a5 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:09.468678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:09.469307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:09.469591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:09.469863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:09.470608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:09.470914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:09.472628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:09.493935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 613/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:09.500799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:09.613511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a65b9-6025-414c-b2f5-63e489b62723 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:09.651115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:09.651789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:09.652074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:09.652404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:09.683645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:09.724866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:09.726602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:09.842426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 601/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:42:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:42:09.848530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:09.869189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:09.980771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b9eaf-7442-4236-a160-d0c39eeedc54 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:10.019687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:10.020482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:10.020897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:10.021259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:10.032334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:10.033910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:10.049970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:10.051279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:10.052479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:10.052479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:10.052479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:10.058706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 614/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:42:10.060064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:10.134686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:10.186348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b7e15-eefd-4fdb-b997-a0dd91c49121 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:10.230846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:10.232533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:10.232932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:10.233352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:10.234111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:10.234479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:10.236027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:10.253643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 602/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:10.255092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:10.342382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:10.384367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594dc35-41c9-4e4e-9c4f-8063c295d586 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:10.429680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:10.430448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:10.430763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:10.431125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:10.439635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:10.441433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:10.455581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:10.457019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:10.458174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:10.461738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 615/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:10.463538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:10.586591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9b4f-9429-4a23-87bf-46275d4c44f7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:10.613447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:10.627299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:10.627912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:10.628197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:10.628469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:10.629221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:10.629529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:10.630954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e15cc-10c9-45ba-8432-2fe1ff7c7147 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:10.645070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 603/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:10.652629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:10.770455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e2110-4618-415c-8678-b9cdabf3b3de None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:10.800468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:10.806538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:10.807145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:10.807417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:10.807686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:10.808400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=097f3a7d28294befa98d30e220e01e83) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:10.808784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:10.810892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67447b-c6cb-48bc-8ecb-9ea8f18266ee None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:10.816541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 616/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:42:10 2022] GET /identity/v3/role_assignments?user.id=097f3a7d28294befa98d30e220e01e83&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:12.963432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b034-4f02-4930-9735-45d4fd122881 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:12.964015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b034-4f02-4930-9735-45d4fd122881 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:12.964015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b034-4f02-4930-9735-45d4fd122881 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:12.965370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 604/1220] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:12.970578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:12.971021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:12.971176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:12.989565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:12.990793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:13.062935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 617/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:42:13.332300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b034-4f02-4930-9735-45d4fd122881 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:13.446959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b034-4f02-4930-9735-45d4fd122881 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:13.484725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:13.485363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:13.485647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:13.486042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:13.497460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:13.502240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:13.503199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:13.504122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 605/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:13.509941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:13.624231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6cd02-05aa-4942-9e42-8355418ddcb0 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:13.649426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:13.661219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:13.661847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:13.662202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:13.662479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:13.663144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] RBAC: Adding query filter params (name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:13.663505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:13.664978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:13.674158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 618/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:13.679999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:13.730125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:13.801349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30e5f-7686-43ce-95de-ee020a1cc427 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:13.839674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:13.840833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:13.841111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:13.841379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:13.851236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:13.852720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:13.870442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:13.871521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:13.872560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:13.872560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:13.872560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:13.872560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 606/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:13.879379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:13.985547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ecf3e-b540-4307-93d9-3de479caca24 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.022355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.023220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.023501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.023769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.024401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:14.024735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.026099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.040641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 619/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:14.047177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:14.161504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:14.170468 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8cd11-7684-435b-b189-e18f53ee2b28 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.211131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.211914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.212218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.212539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.222393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.224042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:14.236519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:14.237687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:14.238761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 607/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:42:14.244043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:14.297716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:14.357708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e42f45-bf4b-401c-ace4-a9f82ec8eb45 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.396544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.397206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.397761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.398008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.398734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:14.399032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.400447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.421464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 620/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:14.427553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:14.536994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9988b-3eea-47f2-aa21-9c7fdf3fbf6d None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.569989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.570614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.570897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.571174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.582849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.584527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:14.601195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:14.602396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:14.603366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:14.603366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:14.603366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:14.603366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 608/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:14.608336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:14.720411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b32dd-0ec3-4d3c-b35e-659b996c0f0c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.722650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:14.762905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.763571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.763963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.764182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.764961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:14.765398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.766788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.779607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 621/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:14.785771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:14.796655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:14.895922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cac3-21e7-41d7-8e3e-65f2b9224e85 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:14.939509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:14.940250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:14.940560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:14.940850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:14.948697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:14.950510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:14.962276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:14.963183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:14.964347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 609/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:14.970026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:15.080712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c61fa-ec42-4a08-8c66-f7f44debb750 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:15.117519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:15.118123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:15.118396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:15.118658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:15.119336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:15.119633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:15.121075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d9449-494d-43c4-905a-a365e064e7be None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:15.133839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 622/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:15.141265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:15.211684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:15.250547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeab293-a062-4d9c-9b9d-864e757fd463 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:15.291243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:15.291908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:15.292187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:15.292447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/097f3a7d28294befa98d30e220e01e83/roles/0d3e5644cb2e422ca54b0b1a827e0f0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:15.307939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=097f3a7d28294befa98d30e220e01e83, role_id=0d3e5644cb2e422ca54b0b1a827e0f0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:15.309404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420ef41-c45a-4db2-ad28-deb0bcd427e5 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:15.321417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 610/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:42:15 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/097f3a7d28294befa98d30e220e01e83/roles/0d3e5644cb2e422ca54b0b1a827e0f0d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:42:17.670170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956a889-92d2-4cdd-b45d-546a362a948b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:17.670812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956a889-92d2-4cdd-b45d-546a362a948b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:17.670812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956a889-92d2-4cdd-b45d-546a362a948b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:17.672295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 623/1233] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:17.678049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:17.678526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:17.678656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:17.697718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:17.698916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:17.781076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 611/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:18.100271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956a889-92d2-4cdd-b45d-546a362a948b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:18.132215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:18.218235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956a889-92d2-4cdd-b45d-546a362a948b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:18.255901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:18.256867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:18.257272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:18.257552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:18.270393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:18.275418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:18.276622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:18.281786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 624/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:18.283011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:18.335675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:18.397770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7b70-9df9-4f20-80a4-b83d13fd36d2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:18.434399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:18.435013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:18.435331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:18.435657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:18.436389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] RBAC: Adding query filter params (name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:18.436727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:18.438231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:18.446923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 612/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:18.453842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:18.551932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:18.569719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652efc8-a5c7-477a-a76e-e201a446c254 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:18.610922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:18.611554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:18.611829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:18.612097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:18.622237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:18.623710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:18.639473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:18.640703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:18.641903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:18.641903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:18.641903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:18.644941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 625/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 20 04:42:18.646341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:18.756749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6cebc-990d-4c59-8a4f-d6f0b90486ea None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:18.767443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:18.796673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:18.797418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:18.797776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:18.798174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:18.798923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:18.799271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:18.800936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:18.815448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 613/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:18.821257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:18.927952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9d315-428f-4455-a073-23ef9b724a31 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:18.966681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:18.967357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:18.967632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:18.967901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:18.971125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:18.977464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:18.979101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:18.991046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:18.992367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:18.993727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 626/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:19.000930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:19.122560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cd92-36e5-439e-8483-890a9aeb3959 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:19.165346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:19.165951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:19.166216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:19.166475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:19.167258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:19.167541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:19.169293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:19.192735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 614/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:19.198459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:19.318351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ddf16-6785-4c21-b6cb-aa4ad9186e43 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:19.366875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:19.367488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:19.367765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:19.368036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:19.437379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:19.444237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:19.445716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:19.550139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 627/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:42:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:42:19.556054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:19.587151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:19.676204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e2172-34ab-4f45-819e-1e025acb74e7 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:19.715431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:19.716129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:19.716568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:19.716950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:19.728589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:19.730057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:19.745306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:19.746494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:19.747474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:19.747474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:19.747474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:19.747474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 615/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:19.753337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:19.862744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:19.868906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765ced-ff6f-4e38-9be1-9c51364ab983 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:19.908486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:19.909297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:19.909730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:19.910036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:19.910921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:19.911221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:19.912983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:19.926667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 628/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:19.932786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:20.028878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:20.055136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43915-33fb-4eb6-8b47-4fd837d423ce None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:20.096554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:20.097226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:20.097547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:20.097888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:20.105125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:20.106893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:20.118806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:20.119923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:20.120931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 616/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:20.129345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:20.239729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af497-1002-4239-a554-958e6e41d8ea None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:20.275665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:20.276266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:20.276541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:20.276837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:20.278167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:20.278459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:20.279922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803f36-aadb-478f-8d6a-49e82f69ad68 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:20.293754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 629/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:20.304771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:20.443122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c0dcc-f1bf-45b7-90b3-899c30c60700 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:20.481215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:20.481815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:20.482091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:20.482359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:20.483074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] RBAC: Adding query filter params (role.id=0d3e5644cb2e422ca54b0b1a827e0f0d, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=097f3a7d28294befa98d30e220e01e83) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:20.483375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:20.484815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e2183-048d-4d6c-9170-7a615fb7280b None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:20.490991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 617/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:42:20 2022] GET /identity/v3/role_assignments?user.id=097f3a7d28294befa98d30e220e01e83&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=0d3e5644cb2e422ca54b0b1a827e0f0d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:23.105160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0be0-606c-41cd-8f12-1a8d48188e23 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:23.105796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0be0-606c-41cd-8f12-1a8d48188e23 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:23.105796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0be0-606c-41cd-8f12-1a8d48188e23 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:23.107479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 630/1247] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:42:23.115158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629374a3-6726-4147-9741-d606e59a79ef None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:23.115645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629374a3-6726-4147-9741-d606e59a79ef None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:23.115763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629374a3-6726-4147-9741-d606e59a79ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:23.122482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:23.135238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-629374a3-6726-4147-9741-d606e59a79ef None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:23.136452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-629374a3-6726-4147-9741-d606e59a79ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:23.209773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 618/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 20 04:42:23.510603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0be0-606c-41cd-8f12-1a8d48188e23 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:23.529597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:23.616424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0be0-606c-41cd-8f12-1a8d48188e23 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:23.649984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:23.650609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:23.650893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:23.651172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:23.661631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:23.666399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:23.667545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:23.669233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 631/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:23.674292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629374a3-6726-4147-9741-d606e59a79ef None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:23.788901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629374a3-6726-4147-9741-d606e59a79ef None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:23.819837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:23.825640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:23.826243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:23.826538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:23.826792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:23.827447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:23.827743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:23.829144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:23.841864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 619/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:23.844197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:23.930862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:23.957511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f711e8-7774-4faa-949a-9fe399dd341d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:23.994200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:23.994888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:23.995169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:23.995424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:24.006920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:24.008719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:24.024836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:24.026025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:24.028121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:24.028121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:24.028121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:24.028121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 632/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:24.033700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:24.161060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70ba6d-7ca8-4625-9a3e-c79a36ef5fe8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:24.208717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:24.209353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:24.209634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:24.209905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:24.210554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:24.210854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:24.212253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:24.226029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 620/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:24.232552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:24.336973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d498101-2e0c-47cf-8d57-9d4209693ff2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:24.365079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:24.374048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:24.374698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:24.374981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:24.375256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:24.383267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:24.384741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:24.396132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:24.397752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:24.398834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 633/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:24.404377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:24.461050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:24.507132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183d0cb-2dd7-4496-a723-f87bb7e2ef66 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:24.541976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:24.542590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:24.543434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:24.543705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:24.544436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:24.544772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:24.546236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:24.565139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 621/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:24.571057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:24.674536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a4d76-bd5f-4ab9-a6e2-ecc64b7281ae None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:24.709468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:24.710073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:24.710352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:24.710626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:24.772771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:24.790780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:24.792334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:24.891268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 634/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:42:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 20 04:42:24.897136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:24.992571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956d6b0-e9fa-4ae1-8029-b90d2e0db168 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:25.005591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:25.026821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:25.027395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:25.027675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:25.027946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:25.037447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:25.038874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:25.052872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:25.053992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:25.055140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:25.055140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:25.055140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:25.055140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 622/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:25.060011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:25.143066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:25.156855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316523a-a415-4c36-a31d-fdf27220daab None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:25.190298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:25.190912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:25.191189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:25.191454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:25.192077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:25.192373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:25.193763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:25.204481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 635/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:25.210177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:25.306895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999a0cf-f104-461a-96e1-b4f3c175e82f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:25.339222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:25.340365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:25.340667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:25.340940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:25.347288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:25.348731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:25.359419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:25.360458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:25.361762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 623/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:25.366163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:25.461174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011a9a-453b-4467-b01a-0d43e68d69c4 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:25.493070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:25.493655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:25.493931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:25.494203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:25.494878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:25.495181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:25.496632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0155b9-a779-415f-af50-ef941ec53740 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:25.508090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 636/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:25.514431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:25.539500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:25.609728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e239c8-ec11-4ba4-8d6b-5fcab0c1d2bb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:25.649477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:25.650069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:25.650353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:25.650622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:25.651338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=097f3a7d28294befa98d30e220e01e83) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:25.651641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:25.653641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a1e18-08dc-4642-a5f1-0f383ceb452d None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:25.658855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 624/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:42:25 2022] GET /identity/v3/role_assignments?user.id=097f3a7d28294befa98d30e220e01e83&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:27.868895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491bb6-741c-4941-a71a-6f1038c4b72b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:27.869490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491bb6-741c-4941-a71a-6f1038c4b72b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:27.869490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74491bb6-741c-4941-a71a-6f1038c4b72b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:27.870746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 637/1261] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:27.875870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:27.876306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:27.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:27.894566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:27.895750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:27.960028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 625/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:28.232946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74491bb6-741c-4941-a71a-6f1038c4b72b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:28.243770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:28.331041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74491bb6-741c-4941-a71a-6f1038c4b72b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:28.362785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:28.363387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:28.363659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:28.363924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:28.374047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:28.378728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:28.379891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:28.381147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 638/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:28.385700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:28.406849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:28.483296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fd423-99d8-432f-8a48-28c31d38f39c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:28.515615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:28.516208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:28.516487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:28.516789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:28.517474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:28.517992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:28.519467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:28.527475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 626/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:42:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:28.533275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:28.597694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:28.630392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f3d78-804b-4b8b-867c-fda24008a928 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:28.662517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:28.663067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:28.663345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:28.663616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:28.673790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:28.675217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:28.689783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:28.690911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:28.691983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:28.691983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:28.691983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:28.691983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 639/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:28.696448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:28.789102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:28.811925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de21aaf-9222-44e6-93b6-4313dcb3ec86 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:28.844804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:28.845402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:28.845684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:28.845958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:28.846590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:28.846890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:28.848252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:28.860077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 627/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:28.865725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:28.961919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94fe8-e2f1-400e-bb1a-c5d545b0f33c None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:28.981765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:28.995893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:28.996504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:28.996820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:28.997093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.005046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.006473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:29.017249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:29.018384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:29.019657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 640/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:29.024399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.120620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b217c40-b26e-4b98-a529-291916ecc4f6 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.154458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.155051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.155656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.155656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.156299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:29.156627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.158039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.166599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:29.175522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 628/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:29.181335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.276108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f7d5-216c-49ec-adca-ac7e7725e342 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.308169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.308801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.309078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.309347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.319252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.320698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:29.334559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:29.335695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:29.336688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:29.336688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:29.336688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:29.336688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 641/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:29.341687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.435543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e205e-823e-457a-b36b-c975cb67b948 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.468017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.468636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.468909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.469178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.469809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:29.470107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.471452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.482097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 629/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:29.487576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.518586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:29.583941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659903e4-da37-454c-99af-787eefad08a2 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.615050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.615654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.615928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.616194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.623003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.624421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:29.635117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:29.636758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:29.637779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 642/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:29.642254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.697853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:29.738577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f1abd-4380-41d9-9ee4-81757744a67f None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.774608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.775206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.775486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.775759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.776436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:29.776908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.778327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ce123-fc31-4e21-b4ba-630a0e41e90c None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.789698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 630/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:29.796218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:29.884184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:29.893045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32320d6-32da-4bf1-81e8-4c4596afe5ac None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:29.925417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:29.926057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:29.926363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:29.926635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] PATH_INFO: `/v3/projects/4a04e187c73948d6947123e45487906b/users/097f3a7d28294befa98d30e220e01e83/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:29.928484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a04e187c73948d6947123e45487906b, user_id=097f3a7d28294befa98d30e220e01e83, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:29.929948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a28871-aaf6-44bc-a5b3-abf2ee01be82 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:29.942287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 643/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Tue Sep 20 04:42:29 2022] PUT /identity/v3/projects/4a04e187c73948d6947123e45487906b/users/097f3a7d28294befa98d30e220e01e83/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:42:32.044803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5e2d-16d0-42de-91b5-0a08bcbd794b None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:32.045367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5e2d-16d0-42de-91b5-0a08bcbd794b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:32.045367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5e2d-16d0-42de-91b5-0a08bcbd794b None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:32.046655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 631/1274] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:32.051732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:32.052481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:32.052786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:32.070592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:32.071756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:32.135051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 644/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:32.403341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a5e2d-16d0-42de-91b5-0a08bcbd794b None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:32.499814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a5e2d-16d0-42de-91b5-0a08bcbd794b None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:32.516654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:32.533378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:32.533982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:32.534256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:32.534524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:32.544077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 20 04:42:32.548837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:32.549941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:32.550840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 632/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Tue Sep 20 04:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:32.555714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:32.631997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:32.650875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fa00a-c7ac-47ef-986d-e233bf87f405 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:32.683569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:32.684099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:32.684379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:32.684669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:32.685314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:32.685607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:32.686952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:32.694561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 645/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Tue Sep 20 04:42:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:32.700769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:32.816405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504dc534-cfbc-45f9-82ee-aa35d7acbc90 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:32.849402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:32.850010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:32.850286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:32.850550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:32.860589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:32.862043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:32.876413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:32.877628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:32.878752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:32.878752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:32.878752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:32.878752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 633/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:32.883404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:32.979080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b7b48-6294-49f1-99bb-95d06d093cfe None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.011722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.012305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.012582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.012877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.013500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:33.013797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.015149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.026745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 646/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:33.032385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:33.067393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:33.135529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aff23-402a-4c55-bf8f-a1aed647baae None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.168188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.168852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.169130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.169399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.176352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.178209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 20 04:42:33.190948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:33.192007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:33.193124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 634/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:33.197967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:33.201773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:33.295808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48156-172d-4218-bbbf-b8d11f1000bf None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.327707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.328299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.328579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.328885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.329599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:33.329903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.331315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.348054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 647/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:33.353780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:33.443862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:33.453378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d653f5-fb25-4c69-b0a9-e170dfdcc97e None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.486237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.486829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.487106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.487371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.551167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.552565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.635101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:33.645418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 635/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 20 04:42:33.652034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:33.709815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:33.749346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c470a9a-6741-48ac-9208-a36f2b7f8a9d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.782448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.783083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.783332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.783597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.793171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.794608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 20 04:42:33.808665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:33.809769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 20 04:42:33.810711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 20 04:42:33.810711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 20 04:42:33.810711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:33.810711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 648/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:33.815367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:33.910604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e7b61-8fa5-476e-b649-c927bfc89cd8 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:33.943471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:33.944056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:33.944332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:33.944628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:33.945256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:33.945553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:33.946897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:33.958077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 636/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:33.963658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:34.059537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f096-a486-4d5c-b7e4-7ba0aeb4bf04 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:34.072324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:34.092190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:34.092840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:34.093117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:34.093387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:34.099681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:34.101147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 20 04:42:34.112848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:34.118523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 04:42:34.119703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 649/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Tue Sep 20 04:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:34.120872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:34.169990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:34.219692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae0507-feb4-4954-bb3f-132f246f0dcb None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:34.252494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:34.253112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:34.253392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:34.253660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:34.254336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:34.254638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:34.256052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08294bf5-4247-448a-affc-b4014353d7ea None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:34.267931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 637/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Tue Sep 20 04:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:34.274420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:34.371408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19d398-81db-430b-af6a-894524c8109d None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:34.403132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:34.403713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:34.403992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:34.404263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:34.405002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] RBAC: Adding query filter params (role.id=6fa16de1683448f59199d3a6ed46c07c, scope.project.id=4a04e187c73948d6947123e45487906b, user.id=097f3a7d28294befa98d30e220e01e83) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:34.405312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:34.406728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce8deb-6965-46d2-b905-f3b87a4f5565 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:34.412346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 650/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Tue Sep 20 04:42:34 2022] GET /identity/v3/role_assignments?user.id=097f3a7d28294befa98d30e220e01e83&scope.project.id=4a04e187c73948d6947123e45487906b&role.id=6fa16de1683448f59199d3a6ed46c07c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:36.600727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210733e7-248d-4147-9915-367042b22111 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:36.601261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210733e7-248d-4147-9915-367042b22111 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:36.601261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210733e7-248d-4147-9915-367042b22111 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:36.602584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 638/1288] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:36.607840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:36.608245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:36.608384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:36.626119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:36.627323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:36.656520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:36.691961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 651/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:36.960816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210733e7-248d-4147-9915-367042b22111 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:37.056484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210733e7-248d-4147-9915-367042b22111 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:37.067140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:37.090588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:37.091199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:37.091477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:37.091894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:37.092725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:37.094315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:37.102912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:37.104136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:37.105269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 639/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:42:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:37.109894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:37.206801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2f442-8e6e-4df3-9706-c4005b5d5c9f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:37.238663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:37.239202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:37.239481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:37.239752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:37.240367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] RBAC: Adding query filter params (name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:37.240696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:37.242055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da880a4-96b7-46af-980a-fdcc65d2959a None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:37.247147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 652/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Sep 20 04:42:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:37.252830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:37.347710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025a6a3-3cb8-44f1-92ea-bffbef565b32 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:37.379797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:37.380373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:37.380674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:37.380943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] PATH_INFO: `/v3/services` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:37.381555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:37.381850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:37.383188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d2ff7e-42a7-4b66-b141-a13ea3ca9128 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:37.387379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 640/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Sep 20 04:42:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:39.506197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3745-2130-4061-a236-6a98b56e4ffc None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:39.506717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3745-2130-4061-a236-6a98b56e4ffc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:39.506717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3745-2130-4061-a236-6a98b56e4ffc None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:39.508066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 653/1293] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:39.513174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2e72-42ca-480d-934f-1199676c0781 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:39.513555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2e72-42ca-480d-934f-1199676c0781 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:39.513717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2e72-42ca-480d-934f-1199676c0781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:39.532100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1afa2e72-42ca-480d-934f-1199676c0781 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:39.533406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1afa2e72-42ca-480d-934f-1199676c0781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:39.561904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:39.598732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 641/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:39.862697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3745-2130-4061-a236-6a98b56e4ffc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:39.893144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:39.962606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3745-2130-4061-a236-6a98b56e4ffc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:39.995538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:39.996343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:39.996647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:39.996927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:39.997582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:39.998935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403545e9-99e7-4c8a-a1fe-de43152e75ae None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:40.013385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 654/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Tue Sep 20 04:42:39 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:42:42.120577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2013d-1d5f-441d-a3b2-696f50e5aa00 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:42.121158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2013d-1d5f-441d-a3b2-696f50e5aa00 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:42.121158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2013d-1d5f-441d-a3b2-696f50e5aa00 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:42.122486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 642/1296] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:42.127534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:42.127903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:42.128072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:42.146126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:42.147543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:42.216983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 655/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:42.480242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2013d-1d5f-441d-a3b2-696f50e5aa00 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:42.578299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2013d-1d5f-441d-a3b2-696f50e5aa00 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:42.611077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:42.611695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:42.611975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:42.612246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:42.612850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:42.614215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:42.623241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:42.624461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:42.625495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 643/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:42:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:42.630035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:42.647776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:42.724706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6b850-3eaa-4426-887f-7d9146fffaa6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:42.763244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:42.763908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:42.764202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:42.764503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:42.765178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:42.765502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:42.766801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:42.772659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 656/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Sep 20 04:42:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:42.778817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:42.812241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:42.884552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0327cc-5646-4563-84ce-36132b3d51a4 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:42.916305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:42.917442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:42.917719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:42.918005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:42.918677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] RBAC: Adding query filter params (interface=public, service_id=da10cad2bb624c128c914640e2861de8, region_id=RegionOne) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:42.918979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:42.920327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4c8c6-430d-4532-b26f-888a5b2fb439 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:42.925134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 644/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Tue Sep 20 04:42:42 2022] GET /identity/v3/endpoints?service_id=da10cad2bb624c128c914640e2861de8&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:42:42.930963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:43.009674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:43.028265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b5a50-1b5a-4ed3-b1f7-2a48e40fd1d6 None admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:43.060966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:43.061557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:43.061833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:43.062099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:43.062686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:43.064038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6e045-c825-4195-bff3-4a6b325fd6c1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:43.070618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 657/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Tue Sep 20 04:42:42 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:42:45.200635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668403cc-7c92-4ebb-b0c5-e4349625dad2 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:45.201123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668403cc-7c92-4ebb-b0c5-e4349625dad2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:45.201219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668403cc-7c92-4ebb-b0c5-e4349625dad2 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:45.202490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 645/1302] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:42:45.207766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:45.208155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:45.208310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:45.225994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:42:45.227183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:42:45.290510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 658/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:42:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 20 04:42:45.556800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668403cc-7c92-4ebb-b0c5-e4349625dad2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:45.647513 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:45.654225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668403cc-7c92-4ebb-b0c5-e4349625dad2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:45.686790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:45.687396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:45.687671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:45.687941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:45.688510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:45.689882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 04:42:45.698237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 04:42:45.699410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application with Lock( Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self._enter() Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application return self.creator() Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application created_value = creator( Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: ERROR keystone.server.flask.application Sep 20 04:42:45.700440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 646/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Tue Sep 20 04:42:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 20 04:42:45.705149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:45.790267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:45.813637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c55f8-9754-4ad6-9d2d-354b3404c9c5 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:45.845286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:45.845892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:45.846156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:45.846421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] PATH_INFO: `/v3/services` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:45.847027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:42:45.847321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:45.848694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a0a63-d27b-4315-b3f2-a6f2d4654df1 None admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:45.853561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 659/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Tue Sep 20 04:42:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:42:45.859981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:42:45.955510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec009a6-7c58-4c49-81ed-7fedf66cece3 None admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:42:45.988043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:42:45.988669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:42:45.988950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:42:45.989222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:42:45.989879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:42:45.991232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21cc3f6-c54d-4cfc-be81-273ab8eab8e9 None admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:42:46.009663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 647/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:42:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:42:55.850740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=30 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:42:56.005531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:49.791507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11ef4-972b-4836-be72-2414141a1f6e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:49.791969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11ef4-972b-4836-be72-2414141a1f6e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:49.792060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11ef4-972b-4836-be72-2414141a1f6e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:49.798794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c20cd7-93d7-4c1f-a85c-e15dccad3a8b None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:49.799289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c20cd7-93d7-4c1f-a85c-e15dccad3a8b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:49.799384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c20cd7-93d7-4c1f-a85c-e15dccad3a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:49.804478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 660/1307] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:43:49 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:43:49.875084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-42c20cd7-93d7-4c1f-a85c-e15dccad3a8b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:43:49.876298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-42c20cd7-93d7-4c1f-a85c-e15dccad3a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:50.117059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 648/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:43:50.405190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:50.405897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:50.406408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:50.408689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 661/1309] 10.222.0.27 () {54 vars in 963 bytes} [Tue Sep 20 04:43:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:43:50.414487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86acd57-44ea-4574-af0a-eba1d79057ea None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:50.414778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86acd57-44ea-4574-af0a-eba1d79057ea None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:50.415051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86acd57-44ea-4574-af0a-eba1d79057ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:50.499224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e86acd57-44ea-4574-af0a-eba1d79057ea None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:43:50.500912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e86acd57-44ea-4574-af0a-eba1d79057ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:50.525656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:50.590143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 649/1310] 10.222.0.27 () {58 vars in 1060 bytes} [Tue Sep 20 04:43:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:43:50.599408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:43:50.601590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:50.752743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d316-a423-4e0a-9811-db8c3e7332c7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:43:50.845057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:43:50.845818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:50.846197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:50.846567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:50.848400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:50.914385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:50.978094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:43:50.979709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e48eb-e222-4464-87ec-66b17f8118bc service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:43:51.000433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 662/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:43:53.549469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:53.549914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:53.550020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:53.551367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 650/1312] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:43:53.557038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4102c-83f4-4a2b-8941-ed2c57470aa3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:53.557441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4102c-83f4-4a2b-8941-ed2c57470aa3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:53.557722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4102c-83f4-4a2b-8941-ed2c57470aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:53.649541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9cd4102c-83f4-4a2b-8941-ed2c57470aa3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:43:53.650748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9cd4102c-83f4-4a2b-8941-ed2c57470aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:53.743901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 663/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:43:54.021562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:43:54.023080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:54.126709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395da90-b8bf-4337-9fe8-3ee7a3720845 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:43:54.196174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:54.198794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:43:54.199425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:54.199700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:54.200000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:54.202141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:54.307822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:43:54.309221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ea61-a19d-4522-9f3e-ca849dd11dff service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:43:54.328053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 651/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:43:56.746962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707db6-a22d-42dd-827d-659164087f88 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:56.747508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707db6-a22d-42dd-827d-659164087f88 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:56.747508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707db6-a22d-42dd-827d-659164087f88 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:56.749154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 664/1315] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:43:56.754460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247a91a-c9b5-4809-990a-3f31d3c2431c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:56.754884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247a91a-c9b5-4809-990a-3f31d3c2431c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:56.755065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247a91a-c9b5-4809-990a-3f31d3c2431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:56.819887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-2247a91a-c9b5-4809-990a-3f31d3c2431c None None] MFA Rules not processed for user `4359f48b5edc49bc97c3e969f9e4a94e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:43:56.821125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2247a91a-c9b5-4809-990a-3f31d3c2431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:56.871683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:56.915717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 652/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:43:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:43:57.214023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:57.214306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:57.214574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:57.215912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 665/1317] 10.222.0.27 () {54 vars in 963 bytes} [Tue Sep 20 04:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:43:57.221842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fe696-596b-49bb-858f-5c0edd49bb6f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:57.222130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fe696-596b-49bb-858f-5c0edd49bb6f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:57.222404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fe696-596b-49bb-858f-5c0edd49bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:57.297551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-237fe696-596b-49bb-858f-5c0edd49bb6f None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:43:57.298768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-237fe696-596b-49bb-858f-5c0edd49bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:57.371461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 653/1318] 10.222.0.27 () {58 vars in 1060 bytes} [Tue Sep 20 04:43:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:43:57.380565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:43:57.382085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:57.391561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:57.483749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096ff8-76af-45a8-83dc-cb8cda88836a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:43:57.555791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:43:57.556382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:43:57.556683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:43:57.556951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:43:57.558486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:43:57.618244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:43:57.663876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:43:57.665322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b472d-2873-415c-b05e-bd7b7fd1cd67 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:43:57.683277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 666/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:43:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:00.743844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:00.744543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:00.744543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:00.745769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 654/1320] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:00.751674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27357b5-d3f2-4c97-a20c-88b4aa75f8d4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:00.752176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27357b5-d3f2-4c97-a20c-88b4aa75f8d4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:00.752322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27357b5-d3f2-4c97-a20c-88b4aa75f8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:00.817361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a27357b5-d3f2-4c97-a20c-88b4aa75f8d4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:00.819095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a27357b5-d3f2-4c97-a20c-88b4aa75f8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:00.914065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 667/1321] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:01.203879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:01.205443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:01.254510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:01.307650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d87737-bf49-4962-8f82-dddd8017cfd2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:01.378878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:01.379466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:01.379740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:01.380009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:01.381619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:01.472456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:01.486241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:01.487678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802b58f-77e5-4833-811d-c160a0cc7e77 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:01.506518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 655/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:04.304861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:04.305433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:04.305433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:04.306747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 668/1323] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:04.312004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc4af-0286-4f3a-a44e-6378ac27c1fe None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:04.312447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc4af-0286-4f3a-a44e-6378ac27c1fe None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:04.312551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cc4af-0286-4f3a-a44e-6378ac27c1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:04.395645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-069cc4af-0286-4f3a-a44e-6378ac27c1fe None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:04.396937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-069cc4af-0286-4f3a-a44e-6378ac27c1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:04.499067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 656/1324] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:04.790350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:04.791856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:04.811491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:04.893896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e6dd-8573-4f1d-9a95-dfa1ca2b8717 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:04.967833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:04.968440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:04.968779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:04.969053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:04.970651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:05.042727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:05.081600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:05.083033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84505db8-7877-4973-81d6-3a3959137c58 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:05.101430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 669/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:07.334736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:07.335297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:07.335297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:07.336651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 657/1326] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:07.341985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df86a-4f56-4f1a-a867-8621074695a5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:07.342429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df86a-4f56-4f1a-a867-8621074695a5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:07.342527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7df86a-4f56-4f1a-a867-8621074695a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:07.407974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3f7df86a-4f56-4f1a-a867-8621074695a5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:07.409126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3f7df86a-4f56-4f1a-a867-8621074695a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:07.514189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 670/1327] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:07.795395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:07.797015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:07.857179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:07.900206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4683-d15d-4aac-9f7e-8493c4f74d27 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:07.974587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:07.975188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:07.975464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:07.975731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:07.977315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:08.085399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:08.087649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:08.089265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e8ba7-7a5f-4e79-b4c3-2016f323e784 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:08.110431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 658/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:10.353487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:10.354016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:10.354016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:10.355315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 671/1329] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:10.360711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23dd7-a092-4e2d-927b-c565e07bc07d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:10.361119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23dd7-a092-4e2d-927b-c565e07bc07d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:10.361279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c23dd7-a092-4e2d-927b-c565e07bc07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:10.425308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-93c23dd7-a092-4e2d-927b-c565e07bc07d None None] MFA Rules not processed for user `4359f48b5edc49bc97c3e969f9e4a94e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:10.426498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-93c23dd7-a092-4e2d-927b-c565e07bc07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:10.524098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 659/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:10.811648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:10.813200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:10.845655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:10.922203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8cd48-fda7-4b76-9fee-81c3133fb403 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:10.997765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:10.998374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:10.998650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:10.998919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:11.000487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:11.078700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:11.110310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:11.111744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ac2fb-ede6-4233-8e27-f8229fa97563 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:11.130385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 672/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:13.517579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:13.517986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:13.518174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:13.520102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 660/1332] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:13.525497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717cf1-5493-46e5-a21a-178f05699e07 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:13.525927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717cf1-5493-46e5-a21a-178f05699e07 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:13.526046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b717cf1-5493-46e5-a21a-178f05699e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:13.592770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0b717cf1-5493-46e5-a21a-178f05699e07 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:13.593997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0b717cf1-5493-46e5-a21a-178f05699e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:13.693797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 673/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:13.977880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:13.979403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:14.049419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:14.083220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31055ba8-fcc0-492d-8cd3-ebd9c8c6f64f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:14.155029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:14.155206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:14.155499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:14.155745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:14.157347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:14.263501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:14.264987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0068e1-e8c1-401a-8fab-edfa38b63efe service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:14.274850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:14.283083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 661/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:16.476058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:16.476620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:16.476620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:16.477884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 674/1335] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:16.483234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40bade-186b-4591-8985-0d49976c08c2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:16.483653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40bade-186b-4591-8985-0d49976c08c2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:16.483797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40bade-186b-4591-8985-0d49976c08c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:16.548383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3d40bade-186b-4591-8985-0d49976c08c2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:16.549616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3d40bade-186b-4591-8985-0d49976c08c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:16.652967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 662/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:16.940170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:16.941723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:16.981461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:17.043874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c290177-b9e6-4196-a6cf-f80487dc6f07 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:17.115131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:17.115724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:17.115996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:17.116264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:17.117839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:17.199967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:17.225928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:17.227342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492f4a7-03e6-4327-97b5-3c5b1cea80de service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:17.245653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 675/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:19.856374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:19.856916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:19.856916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:19.858226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 663/1338] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:19.863549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd90f-d485-4d3d-86ed-95fe3c19f9e4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:19.864464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd90f-d485-4d3d-86ed-95fe3c19f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:19.864570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd90f-d485-4d3d-86ed-95fe3c19f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:19.929515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-54bbd90f-d485-4d3d-86ed-95fe3c19f9e4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:19.930732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-54bbd90f-d485-4d3d-86ed-95fe3c19f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:20.026090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 676/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:20.313662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:20.315181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:20.387995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:20.415615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a674b-7267-43db-b767-0f0fe9c6ffe7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:20.485792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:20.486384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:20.486658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:20.486924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:20.488452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:20.592946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:20.594363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2e00c-6e3d-444c-92fd-266884ac81ce service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:20.611986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 664/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:24.762780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:24.763255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:24.763352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:24.764844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 677/1341] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:24.770384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c10f88-2189-4cef-847a-8e5166cb1e57 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:24.770826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c10f88-2189-4cef-847a-8e5166cb1e57 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:24.770934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c10f88-2189-4cef-847a-8e5166cb1e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:24.776448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:24.839408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e6c10f88-2189-4cef-847a-8e5166cb1e57 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:24.840773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e6c10f88-2189-4cef-847a-8e5166cb1e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:24.946922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 665/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:25.234394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:25.235942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:25.284808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:25.340919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c7b9c-a23a-4332-b0cd-e1ea50824335 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:25.414065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:25.414701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:25.414986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:25.415258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:25.416870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:25.509605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:25.528054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:25.529516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f34ea-9a75-4490-be55-8fcdc5588aaf service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:25.547143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 678/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:28.296579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:28.297136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:28.297136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:28.298499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 666/1344] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:28.304356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275d8f-e4a6-4373-a421-5d51cd286ed3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:28.304779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275d8f-e4a6-4373-a421-5d51cd286ed3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:28.304946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18275d8f-e4a6-4373-a421-5d51cd286ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:28.370906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-18275d8f-e4a6-4373-a421-5d51cd286ed3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:28.372116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-18275d8f-e4a6-4373-a421-5d51cd286ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:28.467647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 679/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:28.754995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:28.756533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:28.844567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:28.864800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31511129-e2ce-4f3e-b4ec-fffffea56fbd None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:28.945323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:28.945949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:28.946231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:28.946498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:28.948065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:29.077277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:29.078726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec04d1a-5ea5-4c27-bd54-ec837d9fd63d service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:29.097846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 667/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:44:33.308417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:33.308935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:33.309189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:33.310700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 680/1347] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:33.316138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d537-0b6e-409d-aeb3-ce2d111b50cf None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:33.317166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d537-0b6e-409d-aeb3-ce2d111b50cf None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:33.317262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2d537-0b6e-409d-aeb3-ce2d111b50cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:33.332754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:33.384588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1cb2d537-0b6e-409d-aeb3-ce2d111b50cf None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:33.385839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1cb2d537-0b6e-409d-aeb3-ce2d111b50cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:33.478313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 668/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:33.762973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:33.764542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:33.823538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:33.869237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25684816-0abb-4fe6-96ce-e02e23808037 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:33.943495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:33.944101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:33.944380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:33.944687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:33.946257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:34.049647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:34.054107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:34.055553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf7f25-35ae-4f57-ad38-7ba9d8fdda9b service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:34.073224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 681/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:38.943074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a697d-b256-4966-aaea-3da265ff0dcd None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:38.943538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a697d-b256-4966-aaea-3da265ff0dcd None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:38.943623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a697d-b256-4966-aaea-3da265ff0dcd None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:38.947014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 669/1350] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:44:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:38.953057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad71d9-e6a9-48d6-86f2-5cef2f481a7a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:38.953809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad71d9-e6a9-48d6-86f2-5cef2f481a7a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:38.954214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad71d9-e6a9-48d6-86f2-5cef2f481a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:39.046475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e6ad71d9-e6a9-48d6-86f2-5cef2f481a7a None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:39.047862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e6ad71d9-e6a9-48d6-86f2-5cef2f481a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:39.210842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 682/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:44:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:43.374910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8465-1966-4487-b2f5-bdd0f3f3f3e4 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:43.375479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8465-1966-4487-b2f5-bdd0f3f3f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:43.375479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca8465-1966-4487-b2f5-bdd0f3f3f3e4 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:43.377273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 670/1352] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:43.382370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeab3-4416-4df4-ade8-c67ad335e30b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:43.382795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeab3-4416-4df4-ade8-c67ad335e30b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:43.382914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eeab3-4416-4df4-ade8-c67ad335e30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:43.440185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:43.463337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-801eeab3-4416-4df4-ade8-c67ad335e30b None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:43.464535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-801eeab3-4416-4df4-ade8-c67ad335e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:43.476742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=59 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:43.574595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 683/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:44:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:45.000075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b796e88-61b3-48aa-ae75-f21a652c4ec3 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:45.000701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b796e88-61b3-48aa-ae75-f21a652c4ec3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:45.000701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b796e88-61b3-48aa-ae75-f21a652c4ec3 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:45.002140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 671/1354] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:45.006860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f310f-2ada-4e9c-b6ca-01b74ff31cdc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:45.007283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f310f-2ada-4e9c-b6ca-01b74ff31cdc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:45.007414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f310f-2ada-4e9c-b6ca-01b74ff31cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:45.084536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-640f310f-2ada-4e9c-b6ca-01b74ff31cdc None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:45.085871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-640f310f-2ada-4e9c-b6ca-01b74ff31cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:45.128527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:45.195905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 684/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:44:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:48.686176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29d79-5613-4e5c-9586-c696d9e4282e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:48.686692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29d79-5613-4e5c-9586-c696d9e4282e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:48.686692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29d79-5613-4e5c-9586-c696d9e4282e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:48.688100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 672/1356] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:48.694494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a42af-7efc-403b-9530-5e2bd6dc48b0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:48.694943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a42af-7efc-403b-9530-5e2bd6dc48b0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:48.695050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a42af-7efc-403b-9530-5e2bd6dc48b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:48.775669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ca1a42af-7efc-403b-9530-5e2bd6dc48b0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:48.777314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ca1a42af-7efc-403b-9530-5e2bd6dc48b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:48.890838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 685/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:49.195452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:49.195959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:49.196423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:49.198193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 673/1358] 10.222.0.27 () {54 vars in 965 bytes} [Tue Sep 20 04:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:49.204977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bb5c-b0df-4db7-8a63-905c19fa73a0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:49.205502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bb5c-b0df-4db7-8a63-905c19fa73a0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:49.205854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3bb5c-b0df-4db7-8a63-905c19fa73a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:49.227019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:49.292035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b1d3bb5c-b0df-4db7-8a63-905c19fa73a0 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:49.293366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b1d3bb5c-b0df-4db7-8a63-905c19fa73a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:49.418033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 686/1359] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Sep 20 04:44:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:44:49.429803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:49.432057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:49.591035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8014a4-3555-4644-a6b2-7f24d5e53c7f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:49.668384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:49.710873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:49.711524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:49.711783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:49.712007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:49.713612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:49.812132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:49.813764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a468dd1-b870-42bb-8240-2e8467ca5f68 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:49.834716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 674/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:52.205183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:52.206304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:52.206532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:52.208182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 687/1361] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:52.213575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f548-6b27-4c89-8daf-654c691aa977 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:52.214034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f548-6b27-4c89-8daf-654c691aa977 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:52.214118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4f548-6b27-4c89-8daf-654c691aa977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:52.282613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-37e4f548-6b27-4c89-8daf-654c691aa977 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:52.284347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-37e4f548-6b27-4c89-8daf-654c691aa977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:52.325204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:52.394900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 675/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:52.690660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:52.692227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:52.766525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:52.865151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fffe79-3d34-4657-9bb8-a60df98dbd05 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:53.011525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:53.012442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:53.012889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:53.013311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:53.015881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:53.055654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:53.128776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:53.130384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d823-77ca-4afb-a633-a2550d2151b4 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:53.150348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 688/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:55.699557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c023d-dc6f-403e-ad9c-47108f6fd51b None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:55.700105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c023d-dc6f-403e-ad9c-47108f6fd51b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:55.700105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c023d-dc6f-403e-ad9c-47108f6fd51b None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:55.701413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 676/1364] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:55.707277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636cdd-959f-4cff-a451-e90dd7e32c7f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:55.707714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636cdd-959f-4cff-a451-e90dd7e32c7f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:55.707812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84636cdd-959f-4cff-a451-e90dd7e32c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:55.777727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-84636cdd-959f-4cff-a451-e90dd7e32c7f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:55.778984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-84636cdd-959f-4cff-a451-e90dd7e32c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:55.892646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:55.892646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 689/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:56.176537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:56.176986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:56.177445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:56.179182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 677/1366] 10.222.0.27 () {54 vars in 965 bytes} [Tue Sep 20 04:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:56.184736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483502-2d34-4b42-891b-4f243fb3a27e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:56.185119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483502-2d34-4b42-891b-4f243fb3a27e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:56.185490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12483502-2d34-4b42-891b-4f243fb3a27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:56.264844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-12483502-2d34-4b42-891b-4f243fb3a27e None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:56.266194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-12483502-2d34-4b42-891b-4f243fb3a27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:56.387155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 690/1367] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Sep 20 04:44:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:56.395998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:56.397538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:56.547358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faacb62-0c52-490f-a22f-366ad3679d00 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:56.569086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:56.666791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:56.667402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:56.667672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:56.667948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:56.669534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:56.765314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:56.766925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57292a76-19ce-4444-afc6-3c343dea8d41 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:56.786766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 678/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:44:56.964891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371406-29d8-4af9-b406-6d14a9e29971 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:56.965530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371406-29d8-4af9-b406-6d14a9e29971 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:56.965530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371406-29d8-4af9-b406-6d14a9e29971 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:56.967057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 691/1369] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:56.973244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249ef8-18f1-4b8b-92a3-a74f21e63365 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:56.973547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249ef8-18f1-4b8b-92a3-a74f21e63365 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:56.973842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249ef8-18f1-4b8b-92a3-a74f21e63365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:57.010095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:57.054562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4c249ef8-18f1-4b8b-92a3-a74f21e63365 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:57.055841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4c249ef8-18f1-4b8b-92a3-a74f21e63365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:57.188753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 679/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:57.537714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:57.537714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:57.537714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:57.538019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 692/1371] 10.222.0.27 () {54 vars in 962 bytes} [Tue Sep 20 04:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:57.543414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394ae5-01f2-41de-b526-28591d9a79a1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:57.543806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394ae5-01f2-41de-b526-28591d9a79a1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:57.544179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11394ae5-01f2-41de-b526-28591d9a79a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:57.630039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-11394ae5-01f2-41de-b526-28591d9a79a1 None None] MFA Rules not processed for user `f7cdae17359d4ba2ae149afbbd855ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:57.631373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-11394ae5-01f2-41de-b526-28591d9a79a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:57.679741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:57.712970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 680/1372] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Sep 20 04:44:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:44:57.720589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:57.722181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:57.778429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:57.836311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7356707-13d3-47a8-8ec9-d0a898b8ac79 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:57.929638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:57.930653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:57.931026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:57.931408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:57.933413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:58.058835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:58.079105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:58.082607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f56a2-b69d-43ef-867b-2d77f5e37bfc service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:58.108939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 693/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:44:59.044877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:59.045405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:59.045405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:59.046689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 681/1374] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:44:59.052132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324a3d0-805f-448b-9bad-d072ea7ac6a4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:59.052496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324a3d0-805f-448b-9bad-d072ea7ac6a4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:59.052802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324a3d0-805f-448b-9bad-d072ea7ac6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:59.123675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4324a3d0-805f-448b-9bad-d072ea7ac6a4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:44:59.124981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4324a3d0-805f-448b-9bad-d072ea7ac6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:59.222842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 694/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:44:59.511626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:44:59.513163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:59.674670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b432d-4ba8-453e-b243-b7b5ad372556 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:44:59.732272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:44:59.798252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:44:59.798994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:44:59.799272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:44:59.799542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:44:59.801167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:44:59.904943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:44:59.906554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeeac9-794f-4f1d-adb0-ca72c67ac7bd service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:44:59.924748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 682/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:02.145515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:02.146066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:02.146412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:02.147903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 695/1377] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:02.155629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e8da-5b7e-485c-a40d-afc0a4395384 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:02.156123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e8da-5b7e-485c-a40d-afc0a4395384 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:02.156242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662e8da-5b7e-485c-a40d-afc0a4395384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:02.224516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b662e8da-5b7e-485c-a40d-afc0a4395384 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:02.225721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b662e8da-5b7e-485c-a40d-afc0a4395384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:02.232761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:02.328144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 683/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:02.616941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:02.618589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:02.681603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:02.776754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb3f88-6557-4573-a8b7-2e54830a962e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:02.901138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:02.901661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:02.901934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:02.902198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:02.903823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:02.925316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:02.998921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:03.000500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfc748-8811-4e9b-9712-81319fd0b5ac service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:03.019480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 696/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:05.216268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:05.216891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:05.216986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:05.218454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 684/1380] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:05.223510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11149-a10f-4e7c-a8f5-dc7fefd60f1d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:05.223921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11149-a10f-4e7c-a8f5-dc7fefd60f1d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:05.224126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c11149-a10f-4e7c-a8f5-dc7fefd60f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:05.292769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-87c11149-a10f-4e7c-a8f5-dc7fefd60f1d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:05.293967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-87c11149-a10f-4e7c-a8f5-dc7fefd60f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:05.388819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:05.397873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 697/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:05.691032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:05.692663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:05.838407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:05.852064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fe605-8ed4-4ce5-ad7a-4edc9d4b8daf None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:05.965852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:05.966384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:05.966662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:05.966932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:05.968572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:06.067580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:06.069522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903d010-7951-4f1c-862a-d444967ca649 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:06.081097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:06.091181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 685/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:08.594266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:08.594923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:08.595021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:08.596784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 698/1383] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:08.602899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6325-0923-4827-942b-addcd89764fc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:08.604184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6325-0923-4827-942b-addcd89764fc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:08.604184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6325-0923-4827-942b-addcd89764fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:08.693016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-925d6325-0923-4827-942b-addcd89764fc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:08.694887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-925d6325-0923-4827-942b-addcd89764fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:08.792392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 686/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:09.169096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:09.171073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:09.352474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6beb-4227-45cf-b582-61d206c3ed6e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:09.434062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:09.482237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:09.483067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:09.483422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:09.483784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:09.486440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:09.601814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:09.603909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5416630-c7cd-4904-a078-82aef163c55e service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:09.627568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 699/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:12.191614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:12.192135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:12.192367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:12.193922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 687/1386] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:12.199338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821fc6-dc17-4895-b03f-e12198cea502 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:12.199768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821fc6-dc17-4895-b03f-e12198cea502 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:12.199870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821fc6-dc17-4895-b03f-e12198cea502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:12.267669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9f821fc6-dc17-4895-b03f-e12198cea502 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:12.269171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9f821fc6-dc17-4895-b03f-e12198cea502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:12.303114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:12.367609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 700/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:12.658341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:12.659838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:12.737487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:12.808334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf27262-7584-4698-91cf-309076e94b20 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:12.921139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:12.921670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:12.921954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:12.922231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:12.924507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:12.968243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:13.019473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:13.021104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf13e1-4b62-473e-b014-25c16d6916aa service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:13.039397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 688/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:14.027329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9ad3-b752-4d53-aa32-8ad7ab1d80c9 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:14.027864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9ad3-b752-4d53-aa32-8ad7ab1d80c9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:14.027864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9ad3-b752-4d53-aa32-8ad7ab1d80c9 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:14.029555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 701/1389] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:14.036952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161c8c-c5c1-4e6b-96aa-b812730dc97e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:14.037407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161c8c-c5c1-4e6b-96aa-b812730dc97e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:14.037506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7161c8c-c5c1-4e6b-96aa-b812730dc97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:14.102142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a7161c8c-c5c1-4e6b-96aa-b812730dc97e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:14.103330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a7161c8c-c5c1-4e6b-96aa-b812730dc97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:14.199945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 689/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:14.505583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:14.506019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:14.506360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:14.507761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 702/1391] 10.222.0.27 () {54 vars in 962 bytes} [Tue Sep 20 04:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:14.513005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac315-c2bf-4c29-919c-942e43c6d656 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:14.513372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac315-c2bf-4c29-919c-942e43c6d656 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:14.513712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ac315-c2bf-4c29-919c-942e43c6d656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:14.530258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:14.589921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-338ac315-c2bf-4c29-919c-942e43c6d656 None None] MFA Rules not processed for user `ef0eb7d5e5b1416b8018e1a5c49a33e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:14.591116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-338ac315-c2bf-4c29-919c-942e43c6d656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:14.670478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 690/1392] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Sep 20 04:45:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:14.679110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:14.680615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:14.795446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994152f-ddde-42dc-83ef-54a036a8ee26 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:14.860092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:14.877003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:14.877609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:14.877882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:14.878154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:14.879710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:14.989251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:14.990586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85075bc-92b4-4779-aeb8-7d1ac5bb592e service glance] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:15.008546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 703/1393] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:15.235084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:15.235084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:15.235725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:15.236829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 691/1394] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:15.253567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd293d8-f3f0-44cc-ba29-1643e8a29956 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:15.255887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd293d8-f3f0-44cc-ba29-1643e8a29956 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:15.256391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd293d8-f3f0-44cc-ba29-1643e8a29956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:15.333911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5cd293d8-f3f0-44cc-ba29-1643e8a29956 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:15.335164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5cd293d8-f3f0-44cc-ba29-1643e8a29956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:15.377862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:15.437976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 704/1395] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:15.731599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:15.733147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:15.778419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133062d-8b85-454e-91b9-b84af2a62808 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:15.778883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133062d-8b85-454e-91b9-b84af2a62808 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:15.779223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133062d-8b85-454e-91b9-b84af2a62808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:15.845398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:15.845636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1133062d-8b85-454e-91b9-b84af2a62808 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:15.846793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1133062d-8b85-454e-91b9-b84af2a62808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:15.892446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fcc3-0be7-4398-8674-2a52dc7a62a3 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:15.937285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 705/1396] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:45:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:15.981190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71d833-5866-43d5-aeba-c8c93342fda8 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:15.981470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71d833-5866-43d5-aeba-c8c93342fda8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:15.981735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71d833-5866-43d5-aeba-c8c93342fda8 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:15.983065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 706/1397] 10.222.0.27 () {54 vars in 943 bytes} [Tue Sep 20 04:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:45:15.989744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1526-1e9e-4dd7-9da7-950015705297 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:15.990048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1526-1e9e-4dd7-9da7-950015705297 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:15.990317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c1526-1e9e-4dd7-9da7-950015705297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:16.011666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:16.012270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:16.012547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:16.012856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:16.014447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:16.032196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:16.068332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-af0c1526-1e9e-4dd7-9da7-950015705297 None None] MFA Rules not processed for user `7355daeb39aa4d8c8c1eb698fede46f4`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:16.069600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-af0c1526-1e9e-4dd7-9da7-950015705297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:16.089113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:16.109557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:16.111693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:16.132026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 692/1398] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:16.179291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 707/1399] 10.222.0.27 () {58 vars in 1040 bytes} [Tue Sep 20 04:45:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:16.190343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:16.191529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:16.288098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d1e80-fe07-4477-8881-04fdfff42504 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:16.394684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:16.396194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:16.397043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:16.397349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:16.397805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:16.399776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:16.483025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:16.484646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdbf96-b620-436a-a17d-d71727473cb1 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:16.493199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 693/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:45:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:45:18.357119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:18.357755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:18.357755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:18.359162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 708/1401] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:18.366021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3550a-382e-42d9-9835-cac7539dd5f7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:18.366496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3550a-382e-42d9-9835-cac7539dd5f7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:18.366590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3550a-382e-42d9-9835-cac7539dd5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:18.443995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-22d3550a-382e-42d9-9835-cac7539dd5f7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:18.445164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-22d3550a-382e-42d9-9835-cac7539dd5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:18.548949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:18.558790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 694/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:18.849864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:18.851591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:18.955438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:19.015841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b0624-fcf0-4248-a25f-86dd6c8c7c6b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:19.155395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:19.156535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:19.156535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:19.156535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:19.158239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:19.221159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:19.259564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:19.261297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea643de-f30f-43ea-bf0d-be219bc76fa3 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:19.280414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 709/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:21.543850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:21.544442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:21.544442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:21.545782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 695/1404] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:21.551173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5aa8-af53-4ae1-96c2-2c06ffebb4be None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:21.551593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5aa8-af53-4ae1-96c2-2c06ffebb4be None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:21.551705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5aa8-af53-4ae1-96c2-2c06ffebb4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:21.627250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-51cf5aa8-af53-4ae1-96c2-2c06ffebb4be None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:21.628509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-51cf5aa8-af53-4ae1-96c2-2c06ffebb4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:21.745391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 710/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:22.057096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:22.058616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:22.233566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c591f01-ccf3-4b38-af6e-2fafc491f326 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:22.307027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:22.357320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:22.357936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:22.358216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:22.358487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:22.360079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:22.459043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:22.460575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8fe7e-b929-4e06-9f4e-47f7336e2767 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:22.478430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 696/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:45:24.722997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:24.723922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:24.724203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:24.725580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 711/1407] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:24.730862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8424f-6ece-4633-ba1c-cc41cdd24db8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:24.731896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8424f-6ece-4633-ba1c-cc41cdd24db8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:24.732399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8424f-6ece-4633-ba1c-cc41cdd24db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:24.811251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-27c8424f-6ece-4633-ba1c-cc41cdd24db8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:24.812766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-27c8424f-6ece-4633-ba1c-cc41cdd24db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:24.844557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:24.918749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 697/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:25.211290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:25.212871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:25.252833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:25.375877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf447f0-49d5-4b57-bd4c-e44394da3e8a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:25.498041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:25.505517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:25.506127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:25.506408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:25.506677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:25.508269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:25.604119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:25.605849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e6688-3aa2-4cfc-b7c7-0244d3e4a1e1 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:25.625001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 712/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:34.916977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=30 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:35.622856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=48 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:39.198418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2d8e2-ba6c-41c0-afd8-a6696eff9be0 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:39.198990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2d8e2-ba6c-41c0-afd8-a6696eff9be0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:39.198990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2d8e2-ba6c-41c0-afd8-a6696eff9be0 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:39.200287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 698/1410] 10.222.0.27 () {54 vars in 918 bytes} [Tue Sep 20 04:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:39.205794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5088961-de12-45d4-8e65-6f8d9a7f956c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:39.206215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5088961-de12-45d4-8e65-6f8d9a7f956c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:39.206323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5088961-de12-45d4-8e65-6f8d9a7f956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:39.281267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a5088961-de12-45d4-8e65-6f8d9a7f956c None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:39.282477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a5088961-de12-45d4-8e65-6f8d9a7f956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:39.387245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 713/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Sep 20 04:45:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:39.402084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7623ef-59b8-4677-a3d9-13fdf9dc46f5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:39.402386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7623ef-59b8-4677-a3d9-13fdf9dc46f5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:39.402663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7623ef-59b8-4677-a3d9-13fdf9dc46f5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:39.403987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 699/1412] 10.222.0.27 () {54 vars in 918 bytes} [Tue Sep 20 04:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:39.408528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6da7-7a5e-484e-92af-c57edc10d16d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:39.408794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6da7-7a5e-484e-92af-c57edc10d16d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:39.409067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6da7-7a5e-484e-92af-c57edc10d16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:39.484661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:39.487685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-013b6da7-7a5e-484e-92af-c57edc10d16d None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:39.489709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-013b6da7-7a5e-484e-92af-c57edc10d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:39.602149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 714/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Tue Sep 20 04:45:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:42.145626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:42.146193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:42.146193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:42.147503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:42.152886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ea54-8cc1-4c2c-b4a2-6ab1d3b678c4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:42.153336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ea54-8cc1-4c2c-b4a2-6ab1d3b678c4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:42.153442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ea54-8cc1-4c2c-b4a2-6ab1d3b678c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:42.218356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6597ea54-8cc1-4c2c-b4a2-6ab1d3b678c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:42.220183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6597ea54-8cc1-4c2c-b4a2-6ab1d3b678c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:42.274304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:42.315393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:42.602298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:42.603924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:42.867838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585281e-7473-45d1-bebe-98b1e936ee81 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:42.938348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:42.978126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:42.978729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:42.979008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:42.979281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:42.980882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:43.069787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:43.071362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4206dc-5be2-4744-9b11-3c04f39fab21 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:43.088459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 701/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:44.223682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:44.224236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:44.224236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:44.225614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:44.231878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91994-01aa-4f5a-8fbc-f6c2b49a8688 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:44.232291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91994-01aa-4f5a-8fbc-f6c2b49a8688 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:44.232430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91994-01aa-4f5a-8fbc-f6c2b49a8688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:44.306336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0ab91994-01aa-4f5a-8fbc-f6c2b49a8688 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:44.307528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0ab91994-01aa-4f5a-8fbc-f6c2b49a8688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:44.335088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:44.403010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:44.563102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:44.565313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:44.711701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09736-5b93-489b-a221-408b3342cea1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:44.750072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:44.823933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:44.824532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:44.824836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:44.825101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:44.826663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:44.914342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:44.916010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d39e2-a1b1-4ff7-b056-331b8b6fad7f service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:44.934200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 717/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:46.058944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:46.059510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:46.059510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:46.060868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:46.067391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabbe2b-1ae7-40af-84f2-6c3ead356447 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:46.067844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabbe2b-1ae7-40af-84f2-6c3ead356447 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:46.067958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabbe2b-1ae7-40af-84f2-6c3ead356447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:46.121479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:46.145257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8aabbe2b-1ae7-40af-84f2-6c3ead356447 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:46.146475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8aabbe2b-1ae7-40af-84f2-6c3ead356447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:46.241429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:46.391833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:46.393315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:46.531993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8df8d-e01f-4d07-b6c3-39fe6e4447aa None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:46.543569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:46.648590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:46.649148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:46.649420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:46.649685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:46.651247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:46.738907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:46.740491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9da747-1408-446e-9619-b2dde5ab136f service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:46.758406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 704/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:47.859072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:47.859594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:47.859594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:47.860898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:47.867490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1708-10ed-4db7-bbbe-3ea289b59594 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:47.867907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1708-10ed-4db7-bbbe-3ea289b59594 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:47.868071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f1708-10ed-4db7-bbbe-3ea289b59594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:47.894710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:47.951197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6e0f1708-10ed-4db7-bbbe-3ea289b59594 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:47.952392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6e0f1708-10ed-4db7-bbbe-3ea289b59594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:48.045154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:48.195431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:48.196975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:48.307142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:48.459571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197392f-1928-44f7-95e0-a91af861dcff None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:48.575742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:48.576359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:48.576658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:48.576951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:48.578497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:48.652304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:48.665708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:48.667303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45217d45-ff73-4ab2-ba36-a16e20a93bd7 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:48.685397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 720/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:49.787588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:49.788150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:49.788150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:49.789507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:49.795752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bdc8-2846-4a6b-bfa5-592cccc4af03 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:49.796195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bdc8-2846-4a6b-bfa5-592cccc4af03 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:49.796305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bdc8-2846-4a6b-bfa5-592cccc4af03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:49.869749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-41e4bdc8-2846-4a6b-bfa5-592cccc4af03 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:49.870863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-41e4bdc8-2846-4a6b-bfa5-592cccc4af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:49.964877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:45:50.116568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:50.118225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:50.200779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:50.260888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd84df0-2867-434e-ac56-dcd312dc4bd1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:50.370018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:50.370611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:50.370887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:50.371154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:50.372762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:50.419774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:50.459773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:50.461383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b6dbe-5740-4b86-90a7-049e8f7e38fd service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:50.478916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 707/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:51.528164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:51.528755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:51.528755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:51.530079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:51.536773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bcfef-7e09-4032-b990-9e957264dd29 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:51.537232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bcfef-7e09-4032-b990-9e957264dd29 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:51.537345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77bcfef-7e09-4032-b990-9e957264dd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:51.621169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f77bcfef-7e09-4032-b990-9e957264dd29 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:51.622371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f77bcfef-7e09-4032-b990-9e957264dd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:51.715283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:51.868429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:51.869949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:51.920904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:52.016829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fa982-58ba-455a-ad82-05998dba2101 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:52.127381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:52.127889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:52.128170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:52.128440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:52.130061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:52.137770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:52.220873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:52.222451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d908e5-d451-4890-bc0e-5ee7e5b01709 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:52.239702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 723/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:45:53.289850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:53.290420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:53.290420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:53.291735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:53.297901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087987be-a134-4936-835c-21fcab3fdd9e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:53.298311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087987be-a134-4936-835c-21fcab3fdd9e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:53.298435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087987be-a134-4936-835c-21fcab3fdd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:53.372186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-087987be-a134-4936-835c-21fcab3fdd9e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:53.373397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-087987be-a134-4936-835c-21fcab3fdd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:53.441147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:53.468725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:53.619227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:53.620753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:53.637682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:53.763218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad287e28-6245-4b20-9228-b036b11c9075 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:53.849179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:53.874324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:53.874933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:53.875215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:53.875487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:53.877119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:53.964662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:53.966259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8490765-0e9e-4784-acb7-5275c2f4c3c5 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:53.984276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 710/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:55.052788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a9ec-414a-4730-9782-52731133da23 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:55.052788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a9ec-414a-4730-9782-52731133da23 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:55.053466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a9ec-414a-4730-9782-52731133da23 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:55.055179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:55.060955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55770-56c7-46d2-b721-23de376b2235 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:55.061553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55770-56c7-46d2-b721-23de376b2235 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:55.061827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55770-56c7-46d2-b721-23de376b2235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:55.136545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5fe55770-56c7-46d2-b721-23de376b2235 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:55.137778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5fe55770-56c7-46d2-b721-23de376b2235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:55.179153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:55.240970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:55.391193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a9ec-414a-4730-9782-52731133da23 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:55.393186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0518a9ec-414a-4730-9782-52731133da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:55.543570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a9ec-414a-4730-9782-52731133da23 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:55.598300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:55.656012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:55.656537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:55.656842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:55.657111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:55.659272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:55.745477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:55.747044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d22824-ac8d-431b-9104-c7c766e6e8f5 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:55.765450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 726/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:56.818147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:56.818698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:56.818698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:56.820091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:56.826081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fa432-6569-41a9-803e-c0805a216959 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:56.826492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fa432-6569-41a9-803e-c0805a216959 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:56.826629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fa432-6569-41a9-803e-c0805a216959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:56.909160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4c8fa432-6569-41a9-803e-c0805a216959 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:56.910364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4c8fa432-6569-41a9-803e-c0805a216959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:56.912327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:57.004270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:45:57.153155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:45:57.154581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:57.301416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44367ef-1fb7-43ab-8b71-709d9928f989 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:45:57.327658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:57.412261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:45:57.412809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:57.413087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:57.413353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:57.414904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:57.500608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:45:57.502720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af6e71-28cd-45a5-87fe-520f3beb270a service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:45:57.520866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 713/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:45:59.596906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:59.597440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:59.597440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:59.598769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:45:59.603989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf31d-ff02-4c7b-97e4-c7ecc2d4d636 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:45:59.604425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf31d-ff02-4c7b-97e4-c7ecc2d4d636 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:45:59.604524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf31d-ff02-4c7b-97e4-c7ecc2d4d636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:45:59.644854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:45:59.669138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-210cf31d-ff02-4c7b-97e4-c7ecc2d4d636 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:45:59.670322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-210cf31d-ff02-4c7b-97e4-c7ecc2d4d636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:45:59.768676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:00.050450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:00.051960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:00.152646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f64cc-0d93-4176-84c2-ae8cc1991381 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:00.184383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:00.227397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:00.228010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:00.228308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:00.228555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:00.230144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:00.336234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:00.337608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a742c38-27b6-4f4f-aa0f-bc4e4ebd05c2 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:00.355442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 729/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:02.618715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:02.619286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:02.619286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:02.620680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:02.625729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39e8a-a5a1-4dac-a76c-b83e5f5cda07 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:02.626158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39e8a-a5a1-4dac-a76c-b83e5f5cda07 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:02.626281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39e8a-a5a1-4dac-a76c-b83e5f5cda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:02.690876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fbe39e8a-a5a1-4dac-a76c-b83e5f5cda07 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:02.692083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fbe39e8a-a5a1-4dac-a76c-b83e5f5cda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:02.704473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:02.788167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:03.070902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:03.072431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:03.176883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde779c3-2fe9-45a5-98a0-6e987accb81d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:03.190030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:03.249862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:03.250460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:03.250742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:03.251013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:03.253162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:03.366807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:03.368168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3a6e0-4726-4272-b29e-811fb80f7d9c service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:03.386207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 716/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:05.799153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:05.799719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:05.799719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:05.801142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:05.806441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055feec-f652-4b2d-b837-90d43998c93d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:05.806886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055feec-f652-4b2d-b837-90d43998c93d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:05.806993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055feec-f652-4b2d-b837-90d43998c93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:05.854343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:05.872116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6055feec-f652-4b2d-b837-90d43998c93d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:05.873334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6055feec-f652-4b2d-b837-90d43998c93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:05.968215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:06.254470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:06.255990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:06.359065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a399e5-c0dd-4c06-8e07-b707a3509356 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:06.399126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:06.433752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:06.434289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:06.434567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:06.434837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:06.436407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:06.542220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:06.543640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88319c6a-3d56-4bf4-9a8a-c194a044f2b2 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:06.561467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 732/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:08.817600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:08.818147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:08.818147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:08.819475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:08.824696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e476d-eb29-4a49-9382-cfbcba07e1a7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:08.825083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e476d-eb29-4a49-9382-cfbcba07e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:08.825253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e476d-eb29-4a49-9382-cfbcba07e1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:08.890297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f09e476d-eb29-4a49-9382-cfbcba07e1a7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:08.891516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f09e476d-eb29-4a49-9382-cfbcba07e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:08.914517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:08.990559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:09.280211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:09.281861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:09.381968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b3a0e-deed-48de-bf6e-638a5645bdce None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:09.402113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:09.453571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:09.454092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:09.454367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:09.454662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:09.456208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:09.561153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:09.562589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99454a0b-cc86-4bb2-b510-a4af4266a497 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:09.580422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 719/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:11.861468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:11.861979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:11.861979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:11.863268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:11.868735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927ed8-cd55-411a-8ad3-5bd54fb4a4d3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:11.869142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927ed8-cd55-411a-8ad3-5bd54fb4a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:11.869295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927ed8-cd55-411a-8ad3-5bd54fb4a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:11.939017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:11.944206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-34927ed8-cd55-411a-8ad3-5bd54fb4a4d3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:11.945652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-34927ed8-cd55-411a-8ad3-5bd54fb4a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:12.043094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:12.332922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:12.334478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:12.435782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f88679-5178-46ef-acbd-85920ae29fd4 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:12.483309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:12.508090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:12.508714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:12.508992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:12.509258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:12.510813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:12.623193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:12.624639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57b37d-5e15-43b3-9dd1-74068442d70d service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:12.642645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 735/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:14.853743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:14.854287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:14.854287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:14.855661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:14.861599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cfc6-8b03-4209-8740-0673e5206383 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:14.862061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cfc6-8b03-4209-8740-0673e5206383 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:14.862169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cfc6-8b03-4209-8740-0673e5206383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:14.926859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e299cfc6-8b03-4209-8740-0673e5206383 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:14.928059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e299cfc6-8b03-4209-8740-0673e5206383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:14.957984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:15.027778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:15.308475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:15.310409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:15.415045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566668d-61ee-4b6c-8d04-8b350514d43f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:15.443324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:15.488955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:15.489555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:15.489835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:15.490105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:15.491682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:15.598962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:15.600418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158a46f-1f1e-4ea9-8e12-313af1e5d138 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:15.618028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 722/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:17.833302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:17.833816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:17.833816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:17.835172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:17.840496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593524b-03d7-4dca-8b80-3a5b4933f8e8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:17.840940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593524b-03d7-4dca-8b80-3a5b4933f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:17.841092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593524b-03d7-4dca-8b80-3a5b4933f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:17.906672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c593524b-03d7-4dca-8b80-3a5b4933f8e8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:17.907883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c593524b-03d7-4dca-8b80-3a5b4933f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:17.915943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:18.002944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:18.287612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:18.289150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:18.391609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bdec9-37fc-4fba-bd12-7a4a04b54a26 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:18.446448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:18.463756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:18.464354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:18.464655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:18.464929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:18.466496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:18.571899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:18.573339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9fcef-305e-4fe6-8669-9c25350d6824 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:18.591202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 738/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:20.780936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:20.781534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:20.781534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:20.782843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:20.788119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52c81-28aa-4c6f-9136-90225c11d07b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:20.788545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52c81-28aa-4c6f-9136-90225c11d07b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:20.788701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52c81-28aa-4c6f-9136-90225c11d07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:20.853211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9bd52c81-28aa-4c6f-9136-90225c11d07b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:20.854336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9bd52c81-28aa-4c6f-9136-90225c11d07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:20.892320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:20.949472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:21.234846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:21.237083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:21.338288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4390b-e958-4aca-b8bb-cf6f0d917f1e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:21.376878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:21.413878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:21.414402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:21.414679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:21.414948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:21.416498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:21.524199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:21.525824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc191c1-d85e-410c-b16d-482d57cc664b service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:21.543832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 725/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:23.701292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:23.701864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:23.701864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:23.703170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:23.708562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a225066-ba9e-4b69-811d-37b2f923fcf4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:23.709024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a225066-ba9e-4b69-811d-37b2f923fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:23.709135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a225066-ba9e-4b69-811d-37b2f923fcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:23.774715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5a225066-ba9e-4b69-811d-37b2f923fcf4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:23.775915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5a225066-ba9e-4b69-811d-37b2f923fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:23.793835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:23.871708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:24.161778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:24.163287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:24.264531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e3cf-9a9b-488e-b590-def7cf0b42d8 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:24.334456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:24.341402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:24.342003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:24.342277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:24.342544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:24.344099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:24.450063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:24.451386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d58d7-0991-44bf-af0d-1c040a65b3d7 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:24.469133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 741/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:26.625638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:26.626231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:26.626231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:26.627542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:26.632923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0e48-0433-46d2-a618-97ff4794f54c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:26.633317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0e48-0433-46d2-a618-97ff4794f54c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:26.633481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0e48-0433-46d2-a618-97ff4794f54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:26.698875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-259f0e48-0433-46d2-a618-97ff4794f54c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:26.700082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-259f0e48-0433-46d2-a618-97ff4794f54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:26.748016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:26.796536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:27.084027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:27.085582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:27.189166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bb07e-3d83-41dd-8ee7-e36b4639cf35 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:27.235536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:27.261275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:27.261873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:27.262150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:27.262419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:27.263980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:27.370017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:27.371377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6428439-11ae-4c25-83d3-98fb7c7a21f4 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:27.389194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 728/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:36.795021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=22 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:37.386973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=66 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:47.414102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631914b-9e04-4030-b638-d4deee81192c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:47.414637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631914b-9e04-4030-b638-d4deee81192c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:47.414637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631914b-9e04-4030-b638-d4deee81192c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:47.415945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 743/1471] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:47.421034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39fad-22a0-445a-84f3-f31da3c25f0d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:47.421463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39fad-22a0-445a-84f3-f31da3c25f0d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:47.421580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39fad-22a0-445a-84f3-f31da3c25f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:47.485427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-81a39fad-22a0-445a-84f3-f31da3c25f0d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:47.486612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-81a39fad-22a0-445a-84f3-f31da3c25f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:47.580950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 729/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:47.881139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:47.881422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:47.881698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:47.883023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 744/1473] 10.222.0.27 () {54 vars in 962 bytes} [Tue Sep 20 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:47.887878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a558cd5-a1be-4d55-9cc9-bf0aaf0b0fa0 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:47.888160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a558cd5-a1be-4d55-9cc9-bf0aaf0b0fa0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:47.888430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a558cd5-a1be-4d55-9cc9-bf0aaf0b0fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:47.961834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5a558cd5-a1be-4d55-9cc9-bf0aaf0b0fa0 None None] MFA Rules not processed for user `ef0eb7d5e5b1416b8018e1a5c49a33e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:47.963015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5a558cd5-a1be-4d55-9cc9-bf0aaf0b0fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:47.982474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:48.037216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 730/1474] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Sep 20 04:46:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:48.044882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:48.046390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:48.149306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460760b4-f0f2-445b-aa8a-9e7e2afb34a1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:48.226510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:48.227104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:48.227377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:48.227641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:48.229494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:48.283517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:48.337101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:48.338516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d9f93-d7b0-432d-9177-01f0d51db5da service glance] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:48.356952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 745/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:50.562479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:50.562969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:50.563087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:50.564914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 731/1476] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:50.570115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c925a-be2b-4c4d-9b19-28fa4de5b52b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:50.570559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c925a-be2b-4c4d-9b19-28fa4de5b52b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:50.570657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c925a-be2b-4c4d-9b19-28fa4de5b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:50.634780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-860c925a-be2b-4c4d-9b19-28fa4de5b52b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:50.636039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-860c925a-be2b-4c4d-9b19-28fa4de5b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:50.729838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 746/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:46:51.002665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:51.004178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:51.124524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ea350-e3f8-4690-b691-2c5f92e1d0ed None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:51.175183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:51.218832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:51.219925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:51.220201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:51.220466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:51.221129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:51.222823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:46:51.222823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:46:51.222823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:46:51.223953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1a0ba-3a30-4a3a-8b12-fa4f12a48b07 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:51.245350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 732/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Tue Sep 20 04:46:51 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:46:53.422653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:53.423215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:53.423215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:53.424466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 747/1479] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:53.429686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ffddf-8fc3-4ade-a7a9-d2548c12b27d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:53.430118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ffddf-8fc3-4ade-a7a9-d2548c12b27d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:53.430248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ffddf-8fc3-4ade-a7a9-d2548c12b27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:53.507103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b15ffddf-8fc3-4ade-a7a9-d2548c12b27d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:53.508315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b15ffddf-8fc3-4ade-a7a9-d2548c12b27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:53.604042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 733/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:53.901141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:53.902679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:53.912617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:54.046550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be0b0b-b13e-4ad3-ab54-e8ed471609f6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:54.125210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:54.159742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:54.160341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:54.160628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:54.160906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:54.162456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:54.249814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:54.251399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc6984-c6d8-4c35-be9f-cca5212f9b57 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:54.268976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 748/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:56.559233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:56.559795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:56.559795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:56.561146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 734/1482] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:56.566819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82064d-2de8-415c-80bf-4ff054381f0f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:56.567250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82064d-2de8-415c-80bf-4ff054381f0f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:56.567359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82064d-2de8-415c-80bf-4ff054381f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:56.631734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-df82064d-2de8-415c-80bf-4ff054381f0f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:56.632966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-df82064d-2de8-415c-80bf-4ff054381f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:56.678298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:56.728013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 749/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:57.018314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:57.019829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:57.043274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:57.122842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586aff8-edf8-4e60-a9e3-0aa9d7eb0f67 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:46:57.199514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:46:57.200109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:57.200387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:57.200685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:57.202261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:57.275964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:46:57.311010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:46:57.312445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e0c2f-f101-48dd-ab3c-6a51b910a443 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:46:57.330206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 735/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:46:59.533442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:59.534000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:59.534000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:59.535337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 750/1485] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:46:59.540661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a863ed-e385-4dcb-a2ee-d5ab45210de5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:46:59.541085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a863ed-e385-4dcb-a2ee-d5ab45210de5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:46:59.541198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a863ed-e385-4dcb-a2ee-d5ab45210de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:46:59.605800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-90a863ed-e385-4dcb-a2ee-d5ab45210de5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:46:59.607016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-90a863ed-e385-4dcb-a2ee-d5ab45210de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:46:59.700535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 736/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:46:59.989004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:46:59.990517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:00.091364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd701df-0734-42ab-9587-c37311f1746c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:47:00.163216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:00.164758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:47:00.165373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:00.165840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:00.166270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:00.168021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:00.276781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:47:00.278208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84668215-8a36-4f2b-92fb-a6b54dab78bc service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:47:00.296125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 751/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:47:02.519166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:02.519736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:02.519736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:02.521390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 737/1488] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:47:02.526676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2ba4-888a-4667-8196-984be595de22 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:02.527122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2ba4-888a-4667-8196-984be595de22 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:02.527230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a2ba4-888a-4667-8196-984be595de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:02.593099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c59a2ba4-888a-4667-8196-984be595de22 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:47:02.594322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c59a2ba4-888a-4667-8196-984be595de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:02.645241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:02.687088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 752/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:47:02.979183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:47:02.980745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:03.017129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:03.083025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405d9db-3a45-4777-9afe-2719f00ed7c5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:47:03.156147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:47:03.156761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:03.157040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:03.157313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:03.158892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:03.237866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:03.267668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:47:03.270452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196ae1a-e712-4029-829f-3c90cd1c9a3e service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:47:03.289796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 738/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:47:05.596918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:05.597492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:05.597492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:05.598784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 753/1491] 10.222.0.27 () {54 vars in 927 bytes} [Tue Sep 20 04:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:47:05.604046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbfd05-4b9f-4c8a-90a8-705896d50de2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:05.604431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbfd05-4b9f-4c8a-90a8-705896d50de2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:05.604620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbfd05-4b9f-4c8a-90a8-705896d50de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:05.670610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1dbbfd05-4b9f-4c8a-90a8-705896d50de2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:47:05.671817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1dbbfd05-4b9f-4c8a-90a8-705896d50de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:05.767419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 739/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Tue Sep 20 04:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:47:06.055064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:47:06.056609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:06.159925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fcdd0-0ad8-4927-af86-0ff259be3b97 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:47:06.232779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:47:06.233385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:47:06.233671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:47:06.233939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:47:06.235522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:47:06.243921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:06.344097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:47:06.345462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffed90-f444-47f2-89ac-da673a236d15 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:47:06.362712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 754/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:47:15.765535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=82 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:47:16.360492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=50 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:24.767090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:24.767780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:24.767780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:24.842705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:48:24.843898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:24.937004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 740/1494] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:48:24.944875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:24.945373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:24.945479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.019079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:48:25.020201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.112301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 755/1495] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:48:25.116935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.118118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.131288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a415fc-0045-4d9a-816c-2581140c3dc1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:25.192008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:25.225824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:25.226427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:25.226705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:25.226975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.227608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:48:25.227907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:25.228947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:25.228947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:48:25.228947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:48:25.229829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:25.240622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 741/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:48:25.249434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.258558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5fd60-b784-44e2-b2ff-967e1a7a72ba None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:25.319815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:25.351999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:25.352641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:25.352919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:25.353183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.353796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:25.355498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:25.355498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:48:25.355498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:48:25.356759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:25.412936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 756/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:48:25 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:48:25.417147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.418663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.451129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c5c60-3f9c-49a6-9559-9a43630ff424 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:25.542167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:25.542759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:25.543040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:25.543312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.543942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:25.544994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:25.544994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:48:25.544994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:48:25.546207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:25.588071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 742/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:48:25 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:48:25.593088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.594619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.657346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767c374-58dd-49c5-ba60-b6c3b45bb75b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:25.744241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:25.756729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:25.757336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:25.757611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:25.757876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.758510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:25.759516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:25.759516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:48:25.759516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:48:25.760399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:25.774369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 757/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:48:25.778377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.779863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.790932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ae4d8-0236-47df-a89c-0e822d0e30a1 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:25.815331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:25.883960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:25.884483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:25.884790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:25.885056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:25.885687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:25.886684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:25.886684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:48:25.886684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:48:25.887563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983e218-30b8-4a5d-afcc-db169a95e0a8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:25.900364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 743/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:48:25.904673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:25.906206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:25.939630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7be7-454d-4b99-9eb2-f9edd0ba83f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:26.033450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:26.034161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.034366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.034636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] PATH_INFO: `/v3/projects/f6c031ec3685433d9eebfffb30e84e8a/users/834b09d699de4a5d982b2fae6ca7b053/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.056982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c031ec3685433d9eebfffb30e84e8a, user_id=834b09d699de4a5d982b2fae6ca7b053, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:26.057996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:26.057996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:48:26.057996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:48:26.059945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008911b-8cb2-4e44-96fa-a2ec3871b28d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:26.084264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 758/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:48:25 2022] PUT /identity/v3/projects/f6c031ec3685433d9eebfffb30e84e8a/users/834b09d699de4a5d982b2fae6ca7b053/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:48:26.096492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35527-898a-41df-b1f3-7740c4fc04b8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.096817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35527-898a-41df-b1f3-7740c4fc04b8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.097093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35527-898a-41df-b1f3-7740c4fc04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.154404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-89c35527-898a-41df-b1f3-7740c4fc04b8 None None] MFA Rules not processed for user `834b09d699de4a5d982b2fae6ca7b053`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:48:26.156061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-89c35527-898a-41df-b1f3-7740c4fc04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:26.230931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 744/1502] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:48:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:48:26.244814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.245100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.245371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.246693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 759/1503] 10.222.0.27 () {54 vars in 962 bytes} [Tue Sep 20 04:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:48:26.252093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365877-fd2c-441f-84d1-51e698af1fcb None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.252383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365877-fd2c-441f-84d1-51e698af1fcb None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.252716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72365877-fd2c-441f-84d1-51e698af1fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.286985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:26.331545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-72365877-fd2c-441f-84d1-51e698af1fcb None None] MFA Rules not processed for user `f7cdae17359d4ba2ae149afbbd855ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:48:26.332791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-72365877-fd2c-441f-84d1-51e698af1fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:26.409063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 745/1504] 10.222.0.27 () {58 vars in 1059 bytes} [Tue Sep 20 04:48:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:48:26.416688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:26.418226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:26.468156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:26.520966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6724f7-a719-4d78-8203-5bbfb0f444ce None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:26.592693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:26.593316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.593587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.593857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.595401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:26.691613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:26.692991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:26.696335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:26.710836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 760/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:48:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:48:26.756021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eb7a-5446-4715-b001-15d937ac15b2 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.756313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eb7a-5446-4715-b001-15d937ac15b2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.756589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925eb7a-5446-4715-b001-15d937ac15b2 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.757880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 746/1506] 10.222.0.27 () {52 vars in 830 bytes} [Tue Sep 20 04:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:48:26.761398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:26.762596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:26.859450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456e35-2362-4ca0-b184-8c465649474f service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:26.950488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:26.951038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:26.951854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:26.952118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] PATH_INFO: `/v3/users/834b09d699de4a5d982b2fae6ca7b053` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:26.964521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834b09d699de4a5d982b2fae6ca7b053)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:26.965553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:26.965553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:48:26.965553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:48:26.966751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:26.976675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:27.031802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834b09d699de4a5d982b2fae6ca7b053'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:27.044995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834b09d699de4a5d982b2fae6ca7b053'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:27.067356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834b09d699de4a5d982b2fae6ca7b053'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:27.077545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834b09d699de4a5d982b2fae6ca7b053'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:27.078046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834b09d699de4a5d982b2fae6ca7b053'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:27.083884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 761/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:48:26 2022] DELETE /identity/v3/users/834b09d699de4a5d982b2fae6ca7b053 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:48:27.088264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:27.088740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:27.088851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:27.145603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:48:27.147051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:27.220187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:27.233416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 747/1508] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:48:27.242986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:27.244184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:27.346652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083fee8-0054-400e-8b96-dcf9a7183645 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:27.400435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:27.420435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:27.421061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:27.421337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:27.421601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:27.423160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:27.530870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:27.532312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0368d06-aa4c-4f37-bfd2-718ba84e7568 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:27.550164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 762/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:48:27.883870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:48:27.885132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:48:27.897643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b00a92-a6a9-4f4d-9afc-89107feaa75a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:48:27.994271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:48:27.994895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:48:27.995175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:48:27.995448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] PATH_INFO: `/v3/projects/f6c031ec3685433d9eebfffb30e84e8a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:48:28.005605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c031ec3685433d9eebfffb30e84e8a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:48:28.006626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:48:28.006626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:48:28.006626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:48:28.007823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:48:28.065761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c031ec3685433d9eebfffb30e84e8a', 'request_id': 'req-5ec84202-04d7-49da-a546-4efa66cdb89f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:28.071490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c031ec3685433d9eebfffb30e84e8a', 'request_id': 'req-5ec84202-04d7-49da-a546-4efa66cdb89f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:28.082013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c031ec3685433d9eebfffb30e84e8a', 'request_id': 'req-5ec84202-04d7-49da-a546-4efa66cdb89f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:28.111043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] The token cache is being invalidate because project f6c031ec3685433d9eebfffb30e84e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:48:28.111637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:28.111758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5ec84202-04d7-49da-a546-4efa66cdb89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:48:28.113174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 748/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:48:27 2022] DELETE /identity/v3/projects/f6c031ec3685433d9eebfffb30e84e8a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:48:37.547976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=62 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:48:38.108780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:49:53.333778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:53.334375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:53.334375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:53.335697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:49:53.341099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498aad-9fb3-4563-a9d3-446b3c78d972 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:53.341541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498aad-9fb3-4563-a9d3-446b3c78d972 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:53.341680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61498aad-9fb3-4563-a9d3-446b3c78d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:53.421809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-61498aad-9fb3-4563-a9d3-446b3c78d972 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:49:53.422919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-61498aad-9fb3-4563-a9d3-446b3c78d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:49:53.524774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:49:53.676245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:49:53.678303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:49:53.796217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ac57e-efda-4c6f-91ce-e7f4f1fb21a4 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:49:53.801698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:49:53.802297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:53.802577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:53.802846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:53.810630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:49:53.812197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee2a09-320c-47e6-b094-4d98b7aa3e04 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:49:53.821376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 764/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:49:54.935906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e125-0904-460f-9ddc-b50b16b48f4e None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:54.936461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e125-0904-460f-9ddc-b50b16b48f4e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:54.936461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42e125-0904-460f-9ddc-b50b16b48f4e None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:54.937837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:49:54.944313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16a7ed-3972-44ac-b5d1-cb0b399e67bd None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:54.944836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16a7ed-3972-44ac-b5d1-cb0b399e67bd None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:54.944948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16a7ed-3972-44ac-b5d1-cb0b399e67bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:54.965752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1c16a7ed-3972-44ac-b5d1-cb0b399e67bd None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:49:54.966977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1c16a7ed-3972-44ac-b5d1-cb0b399e67bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:49:54.975274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:49:55.005853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cfbf8-0ab5-4beb-aa0b-d1ffd9dbfa50 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:55.006140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cfbf8-0ab5-4beb-aa0b-d1ffd9dbfa50 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:55.006444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cfbf8-0ab5-4beb-aa0b-d1ffd9dbfa50 None None] PATH_INFO: `/v3` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:55.007728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 751/1516] 10.222.0.25 () {54 vars in 925 bytes} [Tue Sep 20 04:49:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:49:55.014782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43e86-ab38-4fdf-ae39-2e89ed3af393 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:55.015011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43e86-ab38-4fdf-ae39-2e89ed3af393 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:55.015286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43e86-ab38-4fdf-ae39-2e89ed3af393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:55.073749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-06d43e86-ab38-4fdf-ae39-2e89ed3af393 None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:49:55.075503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-06d43e86-ab38-4fdf-ae39-2e89ed3af393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:49:55.127420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:49:55.127645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cfbf8-0ab5-4beb-aa0b-d1ffd9dbfa50 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:49:55.143839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cfbf8-0ab5-4beb-aa0b-d1ffd9dbfa50 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:49:55.147711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 766/1517] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Sep 20 04:49:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:49:55.149902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:49:55.150504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:49:55.150782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:49:55.151050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:49:55.160491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:49:55.162187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82591786-e656-4acd-964f-3155a45aca92 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:49:55.171653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 752/1518] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:50:03.812417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1dff3-8883-49b7-84b3-ec133f4420e2 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:03.812925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1dff3-8883-49b7-84b3-ec133f4420e2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:03.813032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1dff3-8883-49b7-84b3-ec133f4420e2 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:03.814391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 767/1519] 10.222.0.27 () {54 vars in 961 bytes} [Tue Sep 20 04:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:03.819317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71f0f-1e40-415a-9093-6db714ebc328 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:03.819799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71f0f-1e40-415a-9093-6db714ebc328 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:03.819898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c71f0f-1e40-415a-9093-6db714ebc328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:03.839198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d4c71f0f-1e40-415a-9093-6db714ebc328 None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:03.840376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d4c71f0f-1e40-415a-9093-6db714ebc328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:03.848778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 753/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Sep 20 04:50:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:03.856924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1dff3-8883-49b7-84b3-ec133f4420e2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:03.869802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1dff3-8883-49b7-84b3-ec133f4420e2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:03.875572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:03.876160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:03.876437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:03.876748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:03.884814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:03.886390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c00819-0a36-45e4-ac1c-1183309c287d service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:03.895751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 768/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:06.643561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7b5e-445c-4594-9737-49ccbd007dad None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:06.644144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7b5e-445c-4594-9737-49ccbd007dad None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:06.644144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d7b5e-445c-4594-9737-49ccbd007dad None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:06.645456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:06.652421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae21e-9821-4f5b-ab13-806e557c7984 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:06.652885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae21e-9821-4f5b-ab13-806e557c7984 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:06.652982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dae21e-9821-4f5b-ab13-806e557c7984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:06.673019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-87dae21e-9821-4f5b-ab13-806e557c7984 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:06.674238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-87dae21e-9821-4f5b-ab13-806e557c7984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:06.681919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:06.834772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d7b5e-445c-4594-9737-49ccbd007dad None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:06.850286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d7b5e-445c-4594-9737-49ccbd007dad None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:06.855698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:06.856278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:06.856554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:06.856867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:06.864822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:06.866392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481a7e2-cd16-4e91-be2a-4997c3dc31be service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:06.875478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 755/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:07.931967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a20d3-26db-4160-a329-be660ef7ab01 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:07.932533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a20d3-26db-4160-a329-be660ef7ab01 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:07.932533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a20d3-26db-4160-a329-be660ef7ab01 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:07.933900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 20 04:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:07.939897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadaa3-9659-4eec-a7e7-f127e399a71e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:07.940329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadaa3-9659-4eec-a7e7-f127e399a71e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:07.940435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aadaa3-9659-4eec-a7e7-f127e399a71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:07.959392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e2aadaa3-9659-4eec-a7e7-f127e399a71e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:07.960565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e2aadaa3-9659-4eec-a7e7-f127e399a71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:07.968911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 20 04:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:08.121594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a20d3-26db-4160-a329-be660ef7ab01 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:08.137907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a20d3-26db-4160-a329-be660ef7ab01 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:08.143332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:08.143930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:08.144208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:08.144480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:08.153471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:08.155078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcf956-7650-4291-a70c-b199c51d0079 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:08.164816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 771/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:12.049678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:12.050216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:12.050216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:12.051587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:12.056806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d03c3-b461-40d2-b064-a228ef5dcf3f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:12.057232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d03c3-b461-40d2-b064-a228ef5dcf3f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:12.057395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d03c3-b461-40d2-b064-a228ef5dcf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:12.077015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3c5d03c3-b461-40d2-b064-a228ef5dcf3f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:12.078208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3c5d03c3-b461-40d2-b064-a228ef5dcf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:12.085881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:12.375686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:12.377460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:12.409759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:50:12.454624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a7bee-b9f5-48ef-aa86-c5784bebc8b4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:12.458167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:12.458768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:12.459048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:12.459320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:12.468584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:12.470051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e759a-a5d0-40f7-9cf9-4e1dff6105cc service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:12.479224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 758/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:14.601036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5abc-816b-437f-971e-fb9dbbee8b3c None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:14.601615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5abc-816b-437f-971e-fb9dbbee8b3c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:14.601615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5abc-816b-437f-971e-fb9dbbee8b3c None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:14.602906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:14.608098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafac7b-026f-43b4-bd82-5efd0bd09266 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:14.608501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafac7b-026f-43b4-bd82-5efd0bd09266 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:14.608661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafac7b-026f-43b4-bd82-5efd0bd09266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:14.628372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5bafac7b-026f-43b4-bd82-5efd0bd09266 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:14.629596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5bafac7b-026f-43b4-bd82-5efd0bd09266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:14.637756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:50:14.940436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5abc-816b-437f-971e-fb9dbbee8b3c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:14.950981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5abc-816b-437f-971e-fb9dbbee8b3c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:14.954467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:14.955053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:14.955328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:14.955596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:14.967951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:14.969405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca0c6c-58d8-46e7-98e0-9c4e12391aa0 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:14.978874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 774/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:18.697805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:18.698898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:18.698898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:18.700241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:18.707827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2805a-6eae-45be-b002-427732f21c51 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:18.708297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2805a-6eae-45be-b002-427732f21c51 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:18.708405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2805a-6eae-45be-b002-427732f21c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:18.733055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-11e2805a-6eae-45be-b002-427732f21c51 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:18.734250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-11e2805a-6eae-45be-b002-427732f21c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:18.742020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:19.045397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:19.047125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:19.125677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcf3cc-78fa-4991-bd23-d51c01c3d7c3 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:19.129223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:19.129853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:19.130131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:19.130400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:19.139910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:19.141369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7da79-2508-4554-b593-801c57a5044e service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:19.150773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 761/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:50:21.282102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:21.282668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:21.282668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:21.283992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 20 04:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:50:21.289229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8eb9-2e19-43ac-b91f-e2ac3a674913 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:21.289667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8eb9-2e19-43ac-b91f-e2ac3a674913 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:21.289752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff8eb9-2e19-43ac-b91f-e2ac3a674913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:21.311430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-58ff8eb9-2e19-43ac-b91f-e2ac3a674913 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:50:21.312668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-58ff8eb9-2e19-43ac-b91f-e2ac3a674913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:21.320512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 20 04:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:50:21.623652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:50:21.625348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:50:21.638456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54843fc4-7c3a-454a-abc1-c6b720760620 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:50:21.641933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:50:21.642519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:50:21.642799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:50:21.643067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:50:21.652696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:50:21.654114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece16d1e-08d7-46fd-a8c5-d1ab7cfe092c service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:50:21.663262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 777/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:50:31.306243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=63 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:50:31.659234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=37 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:43.211295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.214320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.214320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.244870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.246145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.255314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 763/1540] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.265583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.266212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.266212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.292501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.298466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.311799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 778/1541] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.316056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.330121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12595f18-59c4-4755-bf65-96260aba5865 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.341917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.342533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.342813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.343084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.343741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:43.344045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.345685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.363409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 764/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:43.371446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.384083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d3fdf-d7fa-499f-9718-dec771e31aa3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.391012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.391623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.391900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.392168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.392830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.397230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295980b-418b-42b2-a5a4-2c1ac5016112 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.458119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 779/1543] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:43.462131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.472241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.472533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.472822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.478829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58762d9-4267-4544-bf0a-604bd27c434c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.500867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.502069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.604005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.604662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.604947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.605223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.605880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.608092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24004c08-77b1-4830-b4e5-cbf4e15caa90 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.659568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 765/1544] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:43.661681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a74d58-9b85-455d-8e76-7570fab52679 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.662209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a74d58-9b85-455d-8e76-7570fab52679 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.662703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a74d58-9b85-455d-8e76-7570fab52679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.695211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 780/1545] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.696169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.702121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-09a74d58-9b85-455d-8e76-7570fab52679 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.703834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-09a74d58-9b85-455d-8e76-7570fab52679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.712093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099aeee9-c859-48ba-ae4e-3d7a2009eafe None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.723121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.723745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.724023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.724293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.725068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 766/1546] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.726400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af489-3f4e-437b-a29e-79dff40fa1ed None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.726679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af489-3f4e-437b-a29e-79dff40fa1ed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.726947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32af489-3f4e-437b-a29e-79dff40fa1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.730004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.731637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f7b19-7ed3-4a6a-9e73-683407f654b2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.757863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 781/1547] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:43.759219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.759505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.759776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.775581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e32af489-3f4e-437b-a29e-79dff40fa1ed None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.778238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e32af489-3f4e-437b-a29e-79dff40fa1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.801225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 767/1548] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.808283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.808566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.808868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.815509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.817261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.845546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 782/1549] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.846397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.856448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.858954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.864229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfe680-2fb5-4ef4-9e8c-bde9626f8578 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.872687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.873292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.873568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.873835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.874585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.876188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291c18-1aa8-4190-b47d-45d3b803e6e2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.877711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 768/1550] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.878896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.896439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 783/1551] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:43.899092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698ac06-5785-48b5-9a91-d87798d9c7ea None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.901426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e08e-bc82-471f-81de-d438ceb4873e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.902773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e08e-bc82-471f-81de-d438ceb4873e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.904275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e08e-bc82-471f-81de-d438ceb4873e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.910261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.911173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.911673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.912684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.914240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:43.915936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.917922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.930415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 769/1552] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:43.932803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.933936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-aca9e08e-bc82-471f-81de-d438ceb4873e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.935611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-aca9e08e-bc82-471f-81de-d438ceb4873e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.944906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f4dc-5296-4988-afa4-32fe00d9c769 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.946666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 784/1553] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.948632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d865-0098-4d16-b553-ca2a19985291 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.948853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d865-0098-4d16-b553-ca2a19985291 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.949247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d865-0098-4d16-b553-ca2a19985291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.954576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.955674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.956284 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.956817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.957557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:43.957970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:43.960055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:43.973398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 770/1554] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:43.973621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-2d46d865-0098-4d16-b553-ca2a19985291 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:43.975149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2d46d865-0098-4d16-b553-ca2a19985291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:43.977475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:43.984433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 785/1555] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:43.986337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddb36-3f0c-4047-a481-30ef988a91cc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.986337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddb36-3f0c-4047-a481-30ef988a91cc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.986614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fddb36-3f0c-4047-a481-30ef988a91cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:43.990547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485029d-a7da-4087-886b-48b02b762df8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:43.995780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:43.996579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:43.996885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:43.997185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] PATH_INFO: `/v3/projects/72aa7ea3cb69454797bd7956e12f2a5e/users/3a77211762af4cc9b4f0a2f74ac3ae1d/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.006436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-69fddb36-3f0c-4047-a481-30ef988a91cc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.007632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-69fddb36-3f0c-4047-a481-30ef988a91cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.012012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72aa7ea3cb69454797bd7956e12f2a5e, user_id=3a77211762af4cc9b4f0a2f74ac3ae1d, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.013075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:51:44.013075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:51:44.013075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:51:44.015002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.015558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 786/1556] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.016771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94c59-1af9-435b-966b-76a727470f87 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.017173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94c59-1af9-435b-966b-76a727470f87 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.017446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94c59-1af9-435b-966b-76a727470f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.029974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 771/1557] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:43 2022] PUT /identity/v3/projects/72aa7ea3cb69454797bd7956e12f2a5e/users/3a77211762af4cc9b4f0a2f74ac3ae1d/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:44.034235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.038573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-60f94c59-1af9-435b-966b-76a727470f87 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.039753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-60f94c59-1af9-435b-966b-76a727470f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.095548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3db42-f4cd-41c4-880d-40a146dc0d63 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.100114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.100873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.101011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 787/1558] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.101163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.101412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.102042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.103080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:51:44.103080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:51:44.103080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:51:44.104824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfef5a1-7db6-4f7e-8f27-834c56b97f67 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.106306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.106601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.106879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.109709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.131698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.131698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.134971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 772/1559] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.136251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.136531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.136831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.141409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 788/1560] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.141409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.148635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779578eb-534c-4f92-86f6-e9b05f710ae7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.153652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.160244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.161469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.206520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.207124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.207398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.207729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.208277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.210205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cb602-9051-4e2a-924c-b9355c8ba09c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.222734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 773/1561] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.223647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.234106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf0977-d4bb-440c-bfba-0815ed345554 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.241920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.242101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 789/1562] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.242201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.242201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.242201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.242441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3578-80e1-40ab-b4fb-60e24ce1bc4e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.242441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3578-80e1-40ab-b4fb-60e24ce1bc4e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.242589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:44.242676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3578-80e1-40ab-b4fb-60e24ce1bc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.242766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.242766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b93438-b6bc-4bed-b4e4-9eecfed8e35b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.255138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 774/1563] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:44.256368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.256676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.256955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.259734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-091b3578-80e1-40ab-b4fb-60e24ce1bc4e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.260981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-091b3578-80e1-40ab-b4fb-60e24ce1bc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.275540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.276806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.319428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 790/1564] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.320845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.321147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.321404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.338866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 775/1565] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.339106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.340471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.342274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.350509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a16f84-aa9a-4bcc-850e-b3633db5b030 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.350679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 791/1566] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.351397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.355942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.356759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.357040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.357311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.357938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.359849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8f34-a20f-4329-b386-4cb552b21ac9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.361373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99417fae-e893-4d6e-8b67-15173b28b8d6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.365929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.366505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.366781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.367049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.367682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:44.367979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.369009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:51:44.369009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:51:44.369009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:51:44.369921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.382113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 776/1567] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.382805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 792/1568] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:44.383592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.383883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.384059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.384165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.393827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84643-cd11-46b3-a002-74612792e043 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.398949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.399168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.399442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.399709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.400324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.401887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:51:44.401887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:51:44.401887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:51:44.403086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.404373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:44.405607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:44.413749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 777/1569] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:44.414643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.427785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee499ef-561b-42da-b83a-c456f6f3a594 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.431475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 793/1570] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.431475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.433165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.433775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.434060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.434335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.434973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.436920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.439882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f411bb9-6d1c-4522-a062-d75f5930ca65 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.445181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.445765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.446046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.446320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.446951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:44.447255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.448945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.461191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 794/1571] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:44.461998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.473889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 778/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.474951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.505394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.515415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.526461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc0987-d179-4a25-8484-d7ef59988487 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.540088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb0b82-0111-431b-a4f7-2a0c44e9ad87 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.545526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.546132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.546413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.546686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.547330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:44.547634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.549320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.560915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 779/1573] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:44.561792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.572951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81b2d9-affd-49c5-9b7f-24c6cdc331a5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.575465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.576107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.576337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.576620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.577690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.578272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.578557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.578831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.579493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.581101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.585921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.587327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.591986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 780/1574] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:44.592856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.599852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 795/1575] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:44.600532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.602384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075768b-b58d-4807-8d89-93660d097451 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.607756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.608342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.608733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.608924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.609674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.610261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb985ce4-2edd-412b-ba10-594bcbb7bf26 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.611614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.615193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.616135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.616135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.616378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.616949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:44.617243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.618899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.632665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 796/1576] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:44.633523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.648093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 781/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.649363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.654110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062099e-6ee3-4322-abe3-f806c1b73345 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.719868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8ed73-dd8d-4668-b5f8-e972a5bdd00e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.721433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.722184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.722542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.723001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.723669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.725514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.726116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.726279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.726426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.726689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.727373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.729448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.741716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 797/1578] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:44.741716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.750537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d821f4d-0734-4af3-a1e1-1bb805fd02f6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.756963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.757682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.758048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.758455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.759175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.765362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.766480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 782/1579] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.767386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.800344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 798/1580] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.801413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.814176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfbba1-35ea-4172-b3b3-7d5c4703eca0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.844797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b21ec-2ac6-424c-9309-c08816df947f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.850803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.851417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.851861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.852191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.853419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.854953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.876891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.881799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.882541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.882897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.883238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.883950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.886719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:44.887613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 783/1581] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:44.888962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.902130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 799/1582] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:44.906009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:44.913722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:44.919331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b43489-3512-4c53-9ece-63564d040ea6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.992615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:44.993398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:44.993687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:44.993960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:44.994631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:44.995992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956a6e5-4f90-4a8b-896d-79229faf636d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:44.996963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.002702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.003330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.004573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.004878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.005653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.007725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.023496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 800/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:45.024357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.039559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b250ce-d86f-4102-9773-cd7dd38799d4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.045545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.046165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.046547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.046828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.047639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.049355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.058259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 784/1584] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:45.059181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.063615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 801/1585] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.065146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.083011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c4540-7397-441d-a0d4-1cca2b2296f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.151994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.152649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.152915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.153186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.153832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.155474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.159272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e0243-be84-40c7-98b4-143bf9ef6d5b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.164851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.165701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.166127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.166520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.169852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 802/1586] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.169987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.170073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.170170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.178262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d13a4f-7b21-42b3-9e7a-cdedc29eee64 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.184636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.185283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.185599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.185939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.186892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.188509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.195748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 785/1587] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:45.196961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.209771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382caadb-5a08-486d-9ef5-4ad0bbd29606 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.215002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.215450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 803/1588] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:45.215759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.216101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.216490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] PATH_INFO: `/v3/projects/af4ad4061f914ed98a64da1b0bbfe260/users/c09c4d09e36946fba12b62fd25c5148e/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.219390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.230975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658aff5-40b9-483f-8407-0dc9ef3d02bb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.233534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4ad4061f914ed98a64da1b0bbfe260, user_id=c09c4d09e36946fba12b62fd25c5148e, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.235887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.237180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.238062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.238313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.238719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.239692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.241377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.251601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 786/1589] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:45 2022] PUT /identity/v3/projects/af4ad4061f914ed98a64da1b0bbfe260/users/c09c4d09e36946fba12b62fd25c5148e/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:45.252275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.261291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 804/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.261291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.274673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c107d8-8783-40b6-b99e-b430dd161d57 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.337754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:45.340271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:45.351252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.352264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.352619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.353502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] PATH_INFO: `/v3/projects/87da630024cd4832ac7b09e5845a3d05/users/3dd9016e10cd489caaf388defe62beac/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.364262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ae56e-f795-4bc1-bc1d-e74300bbbb8a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.372860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87da630024cd4832ac7b09e5845a3d05, user_id=3dd9016e10cd489caaf388defe62beac, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.375863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.381642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.382353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.382633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.382973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.383722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.386438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.396483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 805/1591] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:45 2022] PUT /identity/v3/projects/87da630024cd4832ac7b09e5845a3d05/users/3dd9016e10cd489caaf388defe62beac/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:45.397768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.428398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 787/1592] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:45.429807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.442715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f6070-6f35-47fb-9760-d3e7d188dd42 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.485306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b766f0-607d-4e90-8b17-06ae7244dba1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.490948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.491823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.492203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.492628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.494275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.498128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.512035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 806/1593] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.513036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.517155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.517782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.518060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.518332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.518982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.521179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e695158-9e14-4748-a3ee-63b5ae2bea0d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.529123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3c165-fd42-42a9-bc31-c3ff738d62bb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.537088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 788/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.537310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.537426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.537426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.537610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.537725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.537819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.537915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.538914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.540785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.559673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 807/1595] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.560661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.575728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314db0-0a12-4490-9580-2630ec4e7a55 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.582944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.584043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] MFA Rules not processed for user `c09c4d09e36946fba12b62fd25c5148e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:45.584912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.586845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.588140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.588502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:45.589577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.594126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13164b-1334-4e0a-b01f-d8c18a4cff13 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.624250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 808/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.624250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.624250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.624250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.680949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 789/1597] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:45.682302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.695499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] MFA Rules not processed for user `3dd9016e10cd489caaf388defe62beac`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:45.697122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:45.700023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058fd0f-eb41-4983-8145-cdaeb536b14d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.705986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.706707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.707089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.707697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.708422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.710610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.730151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 790/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.731609 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.745249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f251c92-1cc4-41b5-9b57-25f4cd93d1d7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.753682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.754479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.754831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.755217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] PATH_INFO: `/v3/projects/c2376275534c4c0bb4d279a625a1bdcd/users/83fd4adb347749a1ba1df6bf78e19118/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.783821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2376275534c4c0bb4d279a625a1bdcd, user_id=83fd4adb347749a1ba1df6bf78e19118, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.785993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 809/1599] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:45.790560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.792950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.812072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 791/1600] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:45 2022] PUT /identity/v3/projects/c2376275534c4c0bb4d279a625a1bdcd/users/83fd4adb347749a1ba1df6bf78e19118/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:45.823878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.834082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84edf7-d9a3-44c0-bbc8-41686df102d1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.877573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:45.890230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:45.924932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.925564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.925841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.926232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.926769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.928394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.950819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 810/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.952651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:45.955948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969328f8-5ff4-4d40-8b2f-dff7d45c0b2c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.966616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.967247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.967560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.968750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.969414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:45.971065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:45.974863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6e57a-e1cd-4e8c-8527-9b58667bfdc6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:45.983061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:45.984530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:45.985052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:45.985677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] PATH_INFO: `/v3/projects/d885eb63a0134bcc9259a9170f9043d5/users/c6d61b270b704bb68045f11e7f5cb48d/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:45.990419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 792/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:45.990419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.000110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b558b8-af08-4141-91d8-05641d639350 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.005461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.007070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.007453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.007848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.008583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.010685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e221b-562e-4a84-a3e2-8131e8766245 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.014929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d885eb63a0134bcc9259a9170f9043d5, user_id=c6d61b270b704bb68045f11e7f5cb48d, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.018089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.042149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 811/1603] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:45 2022] PUT /identity/v3/projects/d885eb63a0134bcc9259a9170f9043d5/users/c6d61b270b704bb68045f11e7f5cb48d/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:46.043201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.051696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 793/1604] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:45 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:46.053742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47faf-eada-48c9-8394-e857394b06d1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.054138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47faf-eada-48c9-8394-e857394b06d1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.054671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47faf-eada-48c9-8394-e857394b06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.106006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c1e47faf-eada-48c9-8394-e857394b06d1 None None] MFA Rules not processed for user `83fd4adb347749a1ba1df6bf78e19118`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.107530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c1e47faf-eada-48c9-8394-e857394b06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.146784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94f962-738a-47c5-a7a1-1dea4107f7f0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.213959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 794/1605] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:46.215509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e56a-a7a6-410d-bac5-157a88e62825 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.215948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e56a-a7a6-410d-bac5-157a88e62825 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.217323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643e56a-a7a6-410d-bac5-157a88e62825 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.219347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 795/1606] 10.222.0.27 () {54 vars in 965 bytes} [Tue Sep 20 04:51:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:51:46.243878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643e56a-a7a6-410d-bac5-157a88e62825 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.256945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.257698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.258009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.258286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.259133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.261837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.283240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 812/1607] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:46.284024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.294475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78210fff-0cb8-4297-95e6-6aab7e022841 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.299667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.300336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.300634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.300920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] PATH_INFO: `/v3/projects/abdac4af4fb1444c88e98794c0ba4e5e/users/e96bedb5a16b4eae8a72fa806fdc7a06/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.318069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdac4af4fb1444c88e98794c0ba4e5e, user_id=e96bedb5a16b4eae8a72fa806fdc7a06, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.320779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a09f9-18ff-4231-abc7-1a9c9ebbb353 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.346653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 813/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:46 2022] PUT /identity/v3/projects/abdac4af4fb1444c88e98794c0ba4e5e/users/e96bedb5a16b4eae8a72fa806fdc7a06/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:46.352321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.352673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.352892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.390606 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643e56a-a7a6-410d-bac5-157a88e62825 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.397352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] MFA Rules not processed for user `c6d61b270b704bb68045f11e7f5cb48d`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.398794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.418790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:46.454711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:46.487680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.488817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.488817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.489024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] PATH_INFO: `/v3/projects/5f0a24f6f6994f45bd314d3009b0a572/users/49f80b68650940e8850cbd4f84a49ef4/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.566435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 814/1609] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:46.566435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.566435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e495b0b-9a62-428d-9d04-25bca09aa559 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.566435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.567297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.567297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.567297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.567731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0a24f6f6994f45bd314d3009b0a572, user_id=49f80b68650940e8850cbd4f84a49ef4, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.567839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.567839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97434dd-b3be-40f6-b8f7-3f505616acff admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.568024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e034b-bd08-4de7-b857-0f4c7a1760da admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.568024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 796/1610] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:46 2022] PUT /identity/v3/projects/5f0a24f6f6994f45bd314d3009b0a572/users/49f80b68650940e8850cbd4f84a49ef4/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:46.568024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52275be-7e45-488f-8214-a315b473d937 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.568024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52275be-7e45-488f-8214-a315b473d937 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.568024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52275be-7e45-488f-8214-a315b473d937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.568443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 815/1611] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/users => generated 302 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:46.568443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b3f-5185-469a-96c5-5433fbed2f30 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.568443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b3f-5185-469a-96c5-5433fbed2f30 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.568443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510b3f-5185-469a-96c5-5433fbed2f30 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.569105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 816/1612] 10.222.0.27 () {54 vars in 965 bytes} [Tue Sep 20 04:51:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:51:46.570094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510b3f-5185-469a-96c5-5433fbed2f30 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.637105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b52275be-7e45-488f-8214-a315b473d937 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.639598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b52275be-7e45-488f-8214-a315b473d937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.667541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510b3f-5185-469a-96c5-5433fbed2f30 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.678739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.679523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.680179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.680731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] PATH_INFO: `/v3/projects/4f0f0b8842ae496292c27a6ee1350d32/users/c93f22104a7b4a7caf0cc8b7180ffc55/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.700718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0f0b8842ae496292c27a6ee1350d32, user_id=c93f22104a7b4a7caf0cc8b7180ffc55, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.705133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3951-b428-417a-8547-bc7777b90602 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.729219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 817/1613] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:46 2022] PUT /identity/v3/projects/4f0f0b8842ae496292c27a6ee1350d32/users/c93f22104a7b4a7caf0cc8b7180ffc55/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:46.730976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69274-7626-4155-afcb-0cd7b1b4bc7d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.731790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69274-7626-4155-afcb-0cd7b1b4bc7d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.732075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab69274-7626-4155-afcb-0cd7b1b4bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.763228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bab69274-7626-4155-afcb-0cd7b1b4bc7d None None] MFA Rules not processed for user `e96bedb5a16b4eae8a72fa806fdc7a06`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.764378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bab69274-7626-4155-afcb-0cd7b1b4bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.764949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 797/1614] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:46.766226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52275be-7e45-488f-8214-a315b473d937 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.780260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52275be-7e45-488f-8214-a315b473d937 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.842158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 818/1615] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:46.843354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.843636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.843959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.846811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.847441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.847721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.848074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.849427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.851438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b0f7-b37a-4888-ac55-1fff1b23db42 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.866144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.867542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.876698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 819/1616] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:46.877549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:46.889913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0feb-14b1-4f23-baba-975e6e513135 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:46.895546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:46.896429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.896764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.897041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.897646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 798/1617] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:46.897966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:46.899014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.899548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.899654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c0402-ba3d-4565-95cd-dc4c853cd668 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:46.899831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.901195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:46.915950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 820/1618] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:46.915950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:46.915950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:46.915950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:46.922696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.924809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.935600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:46.936847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:46.955548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:47.003013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 821/1619] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:47.004147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.019363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 799/1620] 10.222.0.27 () {58 vars in 1062 bytes} [Tue Sep 20 04:51:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:47.020733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.022221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:47.090308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579a6ad-e530-4505-861b-27a024360a58 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.094788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.095597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.096047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.096486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.104233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8915f-ccf3-4192-aa3b-5cefb7a91101 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.111415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.112494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.112494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.112758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.115281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.116986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.140894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 800/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:47.142229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.165610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1119a9a-f669-4a7d-ad83-e32b2461a258 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.175070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.175792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.176234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.176515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.177350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.187444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.190503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.192362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.203154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 822/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:47.203992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.227982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dc15-feab-44ec-9fb7-6aaf604d742e service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.238823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 801/1623] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:47 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:47.242515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.243200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.243427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.243549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.243802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] PATH_INFO: `/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f/users/38849be4cd2e4df58f11c4ac28d5367f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.271424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ee8dc-20be-4ed4-a76a-4073672eff60 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.280833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27b8ec0878f4542a3902fe2a0f6619f, user_id=38849be4cd2e4df58f11c4ac28d5367f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.284753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.287603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.288235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.288522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.288878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.306352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.308955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 823/1624] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:47 2022] PUT /identity/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f/users/38849be4cd2e4df58f11c4ac28d5367f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:47.309833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.309974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.321357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:47.336808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 802/1625] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:47.338576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.365394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb097-ef91-48f5-80ec-eb043b803dff service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.453820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07affa8f-1ab6-4cbb-a865-476e548ff64d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.472022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.472982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.473432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.474055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.508576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.509403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.513586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.514071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.514914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.517900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.540567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 803/1626] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:47.543342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.574243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44893812-2685-417c-8293-05bb6bd7fae1 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.584921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.586709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.586938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.587268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.589186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.603471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.605575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d77990-8375-4f7a-b8d1-c77285cbbb33 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.606042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.617388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 824/1627] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:47.618333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.629265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5c483-d44e-4834-ab33-8488981db781 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.641850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.642781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.643155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.643526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] PATH_INFO: `/v3/projects/a999b170c9224be6a7c365f8cf941def/users/e1d793209e4643ae8cc42df190025096/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.645457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 804/1628] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:47.647197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ff89-09ab-4628-aff0-c2d814291cd1 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.647472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ff89-09ab-4628-aff0-c2d814291cd1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.647908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ff89-09ab-4628-aff0-c2d814291cd1 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.656198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 805/1629] 10.222.0.27 () {54 vars in 912 bytes} [Tue Sep 20 04:51:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:51:47.656761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ff89-09ab-4628-aff0-c2d814291cd1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.678697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a999b170c9224be6a7c365f8cf941def, user_id=e1d793209e4643ae8cc42df190025096, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.681613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ff89-09ab-4628-aff0-c2d814291cd1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.687739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a33e-1a3c-4ada-9b25-0f365d63c232 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.692063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.693905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.694193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.695854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] PATH_INFO: `/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f/users/38849be4cd2e4df58f11c4ac28d5367f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.702010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27b8ec0878f4542a3902fe2a0f6619f, user_id=38849be4cd2e4df58f11c4ac28d5367f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.708933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.721006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 825/1630] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:47 2022] PUT /identity/v3/projects/a999b170c9224be6a7c365f8cf941def/users/e1d793209e4643ae8cc42df190025096/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:47.725037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f8edc-194f-4391-b4d0-b670dea07cee None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.726477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f8edc-194f-4391-b4d0-b670dea07cee None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.728285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f8edc-194f-4391-b4d0-b670dea07cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.738490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 806/1631] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:47 2022] PUT /identity/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f/users/38849be4cd2e4df58f11c4ac28d5367f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:47.742010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.755740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:47.766868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-634f8edc-194f-4391-b4d0-b670dea07cee None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:47.769203 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-634f8edc-194f-4391-b4d0-b670dea07cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:47.889926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c1c12-694d-440d-9be4-deabb96cf6ee admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:47.912878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:47.913682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.913968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.914272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.914896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:47.916561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:47.926151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 826/1632] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:51:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:47.927946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:47.929052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:47.929512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:47.961223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 807/1633] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:47.962226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:47.970394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:47.975309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] MFA Rules not processed for user `38849be4cd2e4df58f11c4ac28d5367f`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:47.976991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:48.111763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff635-aff9-48c4-bbfa-6c821834c4dc admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:48.123271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:48.124277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.124959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.127395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.139275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 827/1634] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:47 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:48.140132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:48.162783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:48.164281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:48.165720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599a74e-3544-4ddb-b14d-7d2ace9bb33a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:48.177892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:48.180225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.180225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.180225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] PATH_INFO: `/v3/projects/a999b170c9224be6a7c365f8cf941def/users/e1d793209e4643ae8cc42df190025096/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.183901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a999b170c9224be6a7c365f8cf941def, user_id=e1d793209e4643ae8cc42df190025096, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:48.190584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aea2d-593e-4a72-a82c-36c96a5dadfa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:48.194802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 808/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:48.197001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:48.217005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe29c11-2e35-4ba1-8850-bdf76879ab66 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:48.226983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 828/1636] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:48 2022] PUT /identity/v3/projects/a999b170c9224be6a7c365f8cf941def/users/e1d793209e4643ae8cc42df190025096/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:48.229271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6bf13-84ee-4ee1-974e-24f0e09e9c16 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.229548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6bf13-84ee-4ee1-974e-24f0e09e9c16 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.230756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6bf13-84ee-4ee1-974e-24f0e09e9c16 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.238206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 829/1637] 10.222.0.27 () {54 vars in 912 bytes} [Tue Sep 20 04:51:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:51:48.242977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592000c-8c24-49e2-9748-0f65ad44433e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.243236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592000c-8c24-49e2-9748-0f65ad44433e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.243503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592000c-8c24-49e2-9748-0f65ad44433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.310391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e592000c-8c24-49e2-9748-0f65ad44433e None None] MFA Rules not processed for user `e1d793209e4643ae8cc42df190025096`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:48.312030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e592000c-8c24-49e2-9748-0f65ad44433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:48.344295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:48.344942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.345222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.345488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.495731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:48.499941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 830/1638] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:48.508976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.509277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.509558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.529817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:48.544245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:48.574804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 809/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:48 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:48.586239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:48.587665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:48.725607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:48.734290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d458-7e5f-4bf5-9ee2-4f8553b65b29 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:48.745876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:48.746234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.746505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.746775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:48.768417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:48.769917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:48.782565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 831/1640] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:51:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:48.792531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 810/1641] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:48.871391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:48.878012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:48.893870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eafc43-1ece-47d7-99f2-697dbdc03758 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:48.904714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:48.906005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:48.906690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:48.907228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:49.076719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:49.082689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:49.120302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 832/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:49.237116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:49.250333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d76dd2-31bc-4b0a-a4c4-a306243d3fb7 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:49.259062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:49.260593 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:49.262259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:49.262682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:49.287617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:49.295097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:49.326398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 811/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:50.333846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:50.360520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9c3c7-c6dd-4f1c-8f2f-c28c02f030f0 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:50.399484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:50.401245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:50.401665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:50.402107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:50.479186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:50.490530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a19d0-c1cb-4195-a57a-75e55a11b4be service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:50.497804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:50.499140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:50.499617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:50.500035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:50.598807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:50.603371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:50.642009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 833/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:50.752585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:50.752585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a75e-c583-4b09-a013-b65a92c763d6 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:50.784669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 812/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:52.014629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:52.214607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f847-561f-49ff-b33c-a8a8b7f2aec2 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:52.227019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:52.227841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:52.228285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:52.228784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] PATH_INFO: `/v3/users/e96bedb5a16b4eae8a72fa806fdc7a06` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:52.234597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e96bedb5a16b4eae8a72fa806fdc7a06)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:52.236841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:52.328937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:52.348156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e96bedb5a16b4eae8a72fa806fdc7a06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:52.367457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570297c-b3c8-4858-ac8b-5adf5c21a92a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:52.368352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570297c-b3c8-4858-ac8b-5adf5c21a92a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:52.368664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570297c-b3c8-4858-ac8b-5adf5c21a92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:52.370764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e96bedb5a16b4eae8a72fa806fdc7a06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:52.416188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e96bedb5a16b4eae8a72fa806fdc7a06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:52.434941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4570297c-b3c8-4858-ac8b-5adf5c21a92a None None] MFA Rules not processed for user `3a77211762af4cc9b4f0a2f74ac3ae1d`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:52.436313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4570297c-b3c8-4858-ac8b-5adf5c21a92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:52.446343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e96bedb5a16b4eae8a72fa806fdc7a06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:52.449298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fcf7deb6-d439-4d45-8262-0722504a3ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e96bedb5a16b4eae8a72fa806fdc7a06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:52.481554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 834/1646] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:51:52 2022] DELETE /identity/v3/users/e96bedb5a16b4eae8a72fa806fdc7a06 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:52.487745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:52.488299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:52.488796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:52.546359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:52.549400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:52.662845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 813/1647] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:52.672814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:52.675105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:52.675583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:52.693842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 835/1648] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:52.704390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:52.706310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:52.742182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] MFA Rules not processed for user `3a77211762af4cc9b4f0a2f74ac3ae1d`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:52.745460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:52.824903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ca6a9-b015-4f3e-b84f-08cf71165ce3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:52.833633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:52.834640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:52.837263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:52.837706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:52.862696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:52.875574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:52.886068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:52.895583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 814/1649] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:52.910094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:52.913693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:52.921277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 836/1650] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:53.098619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bde6ea-96c7-4600-94b4-3617e2447bee None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:53.126240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:53.126883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:53.127162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:53.127435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:53.150361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:53.156778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:53.172899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:53.174536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30b937-f560-4a62-97f6-0ce44c3d194b service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:53.177606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed23fe-43ad-4cef-b99e-f8a98bd2541c service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:53.194364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:53.195028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:53.195306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:53.197027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] PATH_INFO: `/v3/users/3dd9016e10cd489caaf388defe62beac` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:53.202773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd9016e10cd489caaf388defe62beac)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:53.203912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 815/1651] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:53.209548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:53.348332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd9016e10cd489caaf388defe62beac'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:53.394236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd9016e10cd489caaf388defe62beac'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:53.395679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:53.462648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd9016e10cd489caaf388defe62beac'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:53.484644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd9016e10cd489caaf388defe62beac'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:53.501404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd9016e10cd489caaf388defe62beac'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:53.531121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 837/1652] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:51:53 2022] DELETE /identity/v3/users/3dd9016e10cd489caaf388defe62beac => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:51:53.536193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:53.536694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:53.536779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:53.598471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:53.602816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:53.755396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 816/1653] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:53.776077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:53.777675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:53.956181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb70fc8-1844-4515-b513-30f158462c08 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:53.968719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:53.969365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:53.969650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:53.969997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:53.995169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:53.997219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c0aa3-7bdc-450a-b122-412fc9b6f265 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:54.049855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 838/1654] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:54.295180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:54.296997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:54.328107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229155-9be1-4098-8d49-d7534db8b647 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:54.335474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:54.336693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.337017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.337397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] PATH_INFO: `/v3/projects/abdac4af4fb1444c88e98794c0ba4e5e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.339563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdac4af4fb1444c88e98794c0ba4e5e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:54.349113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:54.375815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59019a-2512-46db-9b9c-2f3778efc315 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.376697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59019a-2512-46db-9b9c-2f3778efc315 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.377590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59019a-2512-46db-9b9c-2f3778efc315 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.378955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 839/1655] 10.222.0.27 () {54 vars in 912 bytes} [Tue Sep 20 04:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:51:54.386987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.387312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.387590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.432262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdac4af4fb1444c88e98794c0ba4e5e', 'request_id': 'req-87ece970-1200-44c4-a8fd-afc845894d0c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:54.439567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdac4af4fb1444c88e98794c0ba4e5e', 'request_id': 'req-87ece970-1200-44c4-a8fd-afc845894d0c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:54.453666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdac4af4fb1444c88e98794c0ba4e5e', 'request_id': 'req-87ece970-1200-44c4-a8fd-afc845894d0c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:54.466552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:54.469698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:54.489100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] The token cache is being invalidate because project abdac4af4fb1444c88e98794c0ba4e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:51:54.489265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:54.491155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-87ece970-1200-44c4-a8fd-afc845894d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:54.497591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 817/1656] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:51:54 2022] DELETE /identity/v3/projects/abdac4af4fb1444c88e98794c0ba4e5e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:54.502365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.502715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.503135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.523442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:54.531662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:54.536262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:54.580900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 840/1657] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:51:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:54.596168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:54.598126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:54.613419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473900c-8d0c-49f3-a6ad-158a406dd58a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:54.625052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:54.625745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.625969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.626240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.640965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:54.643349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0c88c-53a3-4c63-9275-60e1d730ce6f service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:54.655873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 841/1658] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:54.667320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 818/1659] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:54.678455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.679041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.679526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.707084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:54.709453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:54.720503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 842/1660] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:54.725433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:54.738784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5adb-eb29-4fa4-9626-b70e349efc68 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:54.746910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:54.747670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.748078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.748456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.749285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:54.749917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:54.751855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:54.767494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 819/1661] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:51:54.772270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:54.788766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f208bb1-de48-4216-aff1-7a1c06cc8bb5 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:54.804477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:54.805666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.806402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.807233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.808193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:54.810956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:54.854854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 843/1662] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:54 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:54.856502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:54.870482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b187-108a-465f-9dee-c317d588efff admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:54.948347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:54.949118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:54.949571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:54.950199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:54.950776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:54.954418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.007225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 820/1663] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:54 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:55.011660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.028446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61a7c5-89ca-46cc-927a-fa4746116e20 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.039498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.040473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.041061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.041680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.042829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.044974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.060844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 844/1664] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:55.064529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.077441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8656c-89d6-4be8-8b09-0c6a2b6df358 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.085071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.087532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.087532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.087532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.087532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.089926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.110975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 821/1665] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:55.114715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.159577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce5d52-feb5-4469-b521-23da35cb0ad7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.167178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.167980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.168337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.170006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] PATH_INFO: `/v3/projects/19d82685932a4fa0b302195d58e3c7d0/users/f9c3f042a462473d9c9e14b1c67f1870/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.191482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d82685932a4fa0b302195d58e3c7d0, user_id=f9c3f042a462473d9c9e14b1c67f1870, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.194997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e68bf1-b96f-40c1-a685-64cc0394031b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.200756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:55.244219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 845/1666] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:55 2022] PUT /identity/v3/projects/19d82685932a4fa0b302195d58e3c7d0/users/f9c3f042a462473d9c9e14b1c67f1870/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:55.249044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.252494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.253027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.253753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.256232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.311269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bbcf0-7abc-43c6-9b02-431038a1ce24 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.368851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] MFA Rules not processed for user `f9c3f042a462473d9c9e14b1c67f1870`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:55.370218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.465108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.465914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.466421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.466804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] PATH_INFO: `/v3/projects/87da630024cd4832ac7b09e5845a3d05` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.469271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87da630024cd4832ac7b09e5845a3d05)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.489767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 846/1667] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:51:55.493519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.495787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.498605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.532389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:55.601905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87da630024cd4832ac7b09e5845a3d05', 'request_id': 'req-3e021312-1de3-4716-9854-fe7e51bc0ec9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:55.616224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87da630024cd4832ac7b09e5845a3d05', 'request_id': 'req-3e021312-1de3-4716-9854-fe7e51bc0ec9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:55.635231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87da630024cd4832ac7b09e5845a3d05', 'request_id': 'req-3e021312-1de3-4716-9854-fe7e51bc0ec9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:55.651505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] The token cache is being invalidate because project 87da630024cd4832ac7b09e5845a3d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:51:55.651948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:55.652749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:55.654663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 822/1668] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:51:55 2022] DELETE /identity/v3/projects/87da630024cd4832ac7b09e5845a3d05 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:55.655670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.657145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.680529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33657d-a16e-45eb-9012-18abc6b60a7a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.738135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e021312-1de3-4716-9854-fe7e51bc0ec9 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.743579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.744538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.744964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.745344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.746202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.749258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5b087-fdd7-4379-b069-dfb9edd6037d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.791870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 823/1669] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:55 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:55.791870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d08a0-45c6-46ae-b67c-1408c6fbb4b6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.791870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d08a0-45c6-46ae-b67c-1408c6fbb4b6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.791870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d08a0-45c6-46ae-b67c-1408c6fbb4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.808291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:55.810619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.811007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.811986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.816188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.839411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-227d08a0-45c6-46ae-b67c-1408c6fbb4b6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:55.840892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-227d08a0-45c6-46ae-b67c-1408c6fbb4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:55.842295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:55.845638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:55.867556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 847/1670] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:55.868658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:55.880390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae273b-e118-4b6f-9534-e5b82f395b04 service placement] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:55.894957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:55.976229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 824/1671] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:55.985687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:55.986074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:55.987638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:55.999592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.001020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.002182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.003721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.004709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.008393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.049729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:56.053644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:56.068179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 848/1672] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:55 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:56.071747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.077106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 825/1673] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:51:56.078423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.096888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b82df-df62-4984-8993-43a6a4e25730 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.106470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d356ebb-6a82-4f7c-bf1f-56a245135aad None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.122631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.125257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.125257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.125257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.126347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:56.126347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.128697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.139977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.141750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.142249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.142541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.143223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.150110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.183316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 826/1674] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:56.183412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 849/1675] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:56.186493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.188916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.210158 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc60505-a8ea-49e0-a3c8-5fe4c8494473 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.214770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd9091-97b1-438a-8c94-d92e9baf302f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.220535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.221270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.221550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.221827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] PATH_INFO: `/v3/projects/eac402e0f66c43359e9eb030cd550c04/users/917f4c9d694d40e7ac87addc9dabcc48/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.225760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.226539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.226824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.227098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.227755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.237550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.251640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:56.265591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac402e0f66c43359e9eb030cd550c04, user_id=917f4c9d694d40e7ac87addc9dabcc48, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.274803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.291838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 827/1676] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:56 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:56.293137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.303894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 850/1677] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:56 2022] PUT /identity/v3/projects/eac402e0f66c43359e9eb030cd550c04/users/917f4c9d694d40e7ac87addc9dabcc48/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:56.309994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.470915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7138ddd-0d2c-4d3e-bffc-0fe0b1b55c62 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.479662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bed71-f302-4ea8-bd8b-45eaa5d20bf2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.498529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.498698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.499271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.499408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.499539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.499668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.499803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.499928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.500025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.500520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.508359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.514937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.572047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 828/1678] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:51:56 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:56.573652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.581070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 851/1679] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:51:56.582736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.589339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c9bd-c41a-4730-8934-0f1f58ae06a9 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.602299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.602299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.602299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.603035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] PATH_INFO: `/v3/projects/eac402e0f66c43359e9eb030cd550c04/users/917f4c9d694d40e7ac87addc9dabcc48/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.606975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac402e0f66c43359e9eb030cd550c04, user_id=917f4c9d694d40e7ac87addc9dabcc48, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.608349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9811d-87ef-446d-ac0b-26bc4d014dc5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.616192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.617308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.618534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.619054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.619951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.622076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.627439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d2da1-cb0e-459c-bd0a-26008bbebf6a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.640115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 852/1680] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:56.641828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.671001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 829/1681] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:56 2022] PUT /identity/v3/projects/eac402e0f66c43359e9eb030cd550c04/users/917f4c9d694d40e7ac87addc9dabcc48/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:56.673192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.673486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.673764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.687418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377f656-3b24-441e-8692-ba6b827cb920 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.744389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] MFA Rules not processed for user `917f4c9d694d40e7ac87addc9dabcc48`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:56.745760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:56.839052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:56.852003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.852777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.853125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.853645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.854531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.858036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.887025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 853/1682] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:51:56.888525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.901769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0255f-d53e-4f07-b777-483314f9e139 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:56.907853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:56.908915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.909405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.910212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] PATH_INFO: `/v3/projects/219fc1b69bb44df3af1e0f669f0c423b/users/5a9cce1e994949d78af6066042b45869/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:56.931532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=219fc1b69bb44df3af1e0f669f0c423b, user_id=5a9cce1e994949d78af6066042b45869, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:56.935108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db831c-e89b-42f6-8ef2-e99e87ef5a8d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:56.941448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 830/1683] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:56.955409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:56.961127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:56.961655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 854/1684] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:51:56 2022] PUT /identity/v3/projects/219fc1b69bb44df3af1e0f669f0c423b/users/5a9cce1e994949d78af6066042b45869/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:56.975368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:56.975659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:56.975940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:57.035121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] MFA Rules not processed for user `5a9cce1e994949d78af6066042b45869`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:57.036371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:57.097710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:57.107960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609844b-453a-4311-8c77-8e47ff2905f7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:57.225920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 855/1685] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:57.277280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:57.279056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:57.299071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:57.299774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:57.300252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:57.300377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:57.301979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ddf9d-1b7f-4804-a8ce-216f517b05d2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:57.311141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:57.312202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:57.312924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:57.313555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:57.326199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:57.328161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:57.358229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 856/1686] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:51:57.453555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:57.455276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a348a-72fb-4bc6-905c-eb2330821eae service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:57.480676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 831/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:51:58.615964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:58.620786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:58.838811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1819cec-b1c6-4fdf-a8e4-439c003723a1 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:58.855256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:58.856063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:58.856444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:58.856862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] PATH_INFO: `/v3/users/c09c4d09e36946fba12b62fd25c5148e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:58.858837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c09c4d09e36946fba12b62fd25c5148e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:58.861013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:58.895101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c65d4-45e3-42f6-aefa-8dc936f4e110 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:58.895918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c65d4-45e3-42f6-aefa-8dc936f4e110 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:58.895918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c65d4-45e3-42f6-aefa-8dc936f4e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:58.986472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0f0c65d4-45e3-42f6-aefa-8dc936f4e110 None None] MFA Rules not processed for user `49f80b68650940e8850cbd4f84a49ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:58.990470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0f0c65d4-45e3-42f6-aefa-8dc936f4e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.021326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c09c4d09e36946fba12b62fd25c5148e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:59.024777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:59.050840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c09c4d09e36946fba12b62fd25c5148e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:59.085290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c09c4d09e36946fba12b62fd25c5148e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:59.103696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c09c4d09e36946fba12b62fd25c5148e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:59.104928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-936625ad-53b1-4517-a324-cd45cdba7138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c09c4d09e36946fba12b62fd25c5148e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:51:59.129056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 857/1688] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:51:58 2022] DELETE /identity/v3/users/c09c4d09e36946fba12b62fd25c5148e => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:51:59.133967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.134720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.137740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.209535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:59.215782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.300439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 832/1689] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:51:59.311332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.311622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.311899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.360312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] MFA Rules not processed for user `49f80b68650940e8850cbd4f84a49ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:59.361678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.372135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 858/1690] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:59.373338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:59.378454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.418656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:59.460036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 833/1691] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:51:59.461442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:59.463900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.493367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eeb1f-47b5-4a5e-bdaa-9091f2e5dc76 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:59.501765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '917f4c9d694d40e7ac87addc9dabcc48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eac402e0f66c43359e9eb030cd550c04', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:59.503136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.503504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.504033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.506780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:59.507186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:59.509320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:59.526186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 859/1692] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:59.527259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:59.529281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:51:59.572957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa1e4-cdc4-4176-b126-f010ba44890d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:59.577885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:59.579445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.579729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.580002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.602097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:59.604150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:59.628366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 834/1693] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:59.629242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:59.630125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:51:59.646879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ec504-4c84-4428-beeb-8db70192f9b0 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:59.657082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '917f4c9d694d40e7ac87addc9dabcc48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eac402e0f66c43359e9eb030cd550c04', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:59.659828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bb32a-ac4c-419d-b023-2103eed7c532 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:59.662579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.664005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.664443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.665379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:59.668580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35298211-fc23-4ca6-ad75-a986a9ecff5a tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:59.673759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:59.674748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.675208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.675576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.691079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:59.693166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:59.707208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 860/1694] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:51:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:51:59.742407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 835/1695] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:51:59 2022] POST /identity/v3/projects => generated 354 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:51:59.744805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:51:59.776063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7e1f8-d65a-4fe4-b872-ed5e5751a271 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:51:59.896022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.896574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.896574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.910598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '917f4c9d694d40e7ac87addc9dabcc48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eac402e0f66c43359e9eb030cd550c04', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:51:59.911419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:51:59.911704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:51:59.911979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:51:59.913942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:51:59.914269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:51:59.917015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:51:59.950740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 861/1696] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:51:59.989966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] MFA Rules not processed for user `c93f22104a7b4a7caf0cc8b7180ffc55`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:51:59.991431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.012953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:00.035232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9e273-33b8-4d3b-9502-e031a828a8e8 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:00.041001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '917f4c9d694d40e7ac87addc9dabcc48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eac402e0f66c43359e9eb030cd550c04', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:00.041833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:00.042221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:00.042950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] PATH_INFO: `/v3/projects/b29788a0972447de96601cb2893df0ea` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:00.044350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b29788a0972447de96601cb2893df0ea)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:00.046063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: WARNING py.warnings [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:52:00.046063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: warnings.warn(msg) Sep 20 04:52:00.046063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: Sep 20 04:52:00.047568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:00.191806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 836/1697] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:51:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:00.208572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29788a0972447de96601cb2893df0ea', 'request_id': 'req-982bc4cf-9567-46bb-b627-6895655f9056'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.224337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29788a0972447de96601cb2893df0ea', 'request_id': 'req-982bc4cf-9567-46bb-b627-6895655f9056'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.254145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29788a0972447de96601cb2893df0ea', 'request_id': 'req-982bc4cf-9567-46bb-b627-6895655f9056'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.286956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] The token cache is being invalidate because project b29788a0972447de96601cb2893df0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:00.287290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.287927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.289786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 862/1698] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:00 2022] DELETE /identity/v3/projects/b29788a0972447de96601cb2893df0ea => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:00.357678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:00.359212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.360542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:00.362508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.529417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:00.534545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822cdb8-40c0-4a1e-b8db-375798bf5adc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:00.540272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:00.563030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc4cf-9567-46bb-b627-6895655f9056 tempest-VolumeQuotaClassesTest-671620610 tempest-VolumeQuotaClassesTest-671620610-project-admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:00.597708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:00.598815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:00.600810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:00.601259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:00.609547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.748195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:00.748885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:00.749168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:00.749428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] PATH_INFO: `/v3/projects/af4ad4061f914ed98a64da1b0bbfe260` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:00.751101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4ad4061f914ed98a64da1b0bbfe260)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:00.754629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:00.804727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:00.807592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:00.820264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 863/1699] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:00.821021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:00.822559 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.840068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e23866-6fb6-48b4-b1d8-6b2af4f326c9 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:00.847028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:00.848996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:00.849288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:00.849636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] PATH_INFO: `/v3/users/f9c3f042a462473d9c9e14b1c67f1870` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:00.851248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c3f042a462473d9c9e14b1c67f1870)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:00.855349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:00.903516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4ad4061f914ed98a64da1b0bbfe260', 'request_id': 'req-57192cc8-484f-4f5c-be9d-4c5d695decd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.913702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4ad4061f914ed98a64da1b0bbfe260', 'request_id': 'req-57192cc8-484f-4f5c-be9d-4c5d695decd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.935811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4ad4061f914ed98a64da1b0bbfe260', 'request_id': 'req-57192cc8-484f-4f5c-be9d-4c5d695decd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.950344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] The token cache is being invalidate because project af4ad4061f914ed98a64da1b0bbfe260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:00.950867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.952265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:00.953926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 837/1700] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:00 2022] DELETE /identity/v3/projects/af4ad4061f914ed98a64da1b0bbfe260 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:00.957989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:00.972318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:00.994298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c3f042a462473d9c9e14b1c67f1870'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.011741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c3f042a462473d9c9e14b1c67f1870'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.046903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c3f042a462473d9c9e14b1c67f1870'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.081104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c3f042a462473d9c9e14b1c67f1870'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.082812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba799f73-4707-4028-9c69-55ddce673e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c3f042a462473d9c9e14b1c67f1870'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.107476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 864/1701] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:00 2022] DELETE /identity/v3/users/f9c3f042a462473d9c9e14b1c67f1870 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:01.112539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:01.112539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:01.112539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:01.166583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:01.167834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:01.187643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57192cc8-484f-4f5c-be9d-4c5d695decd6 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:01.282474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:01.305060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 865/1702] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:01.316754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:01.323695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:01.331397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:01.332062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:01.332350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:01.332643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] PATH_INFO: `/v3/users/38849be4cd2e4df58f11c4ac28d5367f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:01.334589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38849be4cd2e4df58f11c4ac28d5367f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:01.340115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: WARNING py.warnings [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 20 04:52:01.340115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: warnings.warn(msg) Sep 20 04:52:01.340115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: Sep 20 04:52:01.343896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:01.361224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:01.465899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfd2c2-1b54-4e7b-a441-e14be1b87bf2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:01.476332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:01.477404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:01.477805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:01.478225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:01.497455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:01.500713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094f502-247e-45f3-a251-853c3f73910a service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:01.530193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 866/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:01.552003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae2b99-ec10-4523-b3de-a08412bdde1e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:01.552466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae2b99-ec10-4523-b3de-a08412bdde1e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:01.553006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae2b99-ec10-4523-b3de-a08412bdde1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:01.573560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38849be4cd2e4df58f11c4ac28d5367f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.602827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38849be4cd2e4df58f11c4ac28d5367f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.615318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-15ae2b99-ec10-4523-b3de-a08412bdde1e None None] MFA Rules not processed for user `49f80b68650940e8850cbd4f84a49ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:01.616744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-15ae2b99-ec10-4523-b3de-a08412bdde1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:01.630926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38849be4cd2e4df58f11c4ac28d5367f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.647047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38849be4cd2e4df58f11c4ac28d5367f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.649680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38849be4cd2e4df58f11c4ac28d5367f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:01.678797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 838/1704] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:00 2022] DELETE /identity/v3/users/38849be4cd2e4df58f11c4ac28d5367f => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:01.774197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 867/1705] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:02.325526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:02.327344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:02.516668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:02.517138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:02.517138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:02.656138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31e84e-547d-49bd-a79d-73d39edee041 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:02.671407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:02.673328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:02.676003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:02.678126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:02.679381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:02.680018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] PATH_INFO: `/v3/projects/19d82685932a4fa0b302195d58e3c7d0` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:02.686311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d82685932a4fa0b302195d58e3c7d0)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:02.704931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:02.890925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 868/1706] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:52:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:02.909552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d82685932a4fa0b302195d58e3c7d0', 'request_id': 'req-8e7888f7-6612-4137-a6e8-ad259faf7ca3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:02.993287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d82685932a4fa0b302195d58e3c7d0', 'request_id': 'req-8e7888f7-6612-4137-a6e8-ad259faf7ca3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.037199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d82685932a4fa0b302195d58e3c7d0', 'request_id': 'req-8e7888f7-6612-4137-a6e8-ad259faf7ca3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.038613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:03.090968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] The token cache is being invalidate because project 19d82685932a4fa0b302195d58e3c7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:03.091275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.104353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.111568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 839/1707] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:02 2022] DELETE /identity/v3/projects/19d82685932a4fa0b302195d58e3c7d0 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:03.113399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:03.167606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23dce7-855a-4132-9621-2f74ad19b0fa None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:03.291853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:03.293405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:03.406370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:03.513773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:03.514431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:03.514821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:03.515102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] PATH_INFO: `/v3/users/917f4c9d694d40e7ac87addc9dabcc48` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:03.521917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=917f4c9d694d40e7ac87addc9dabcc48)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:03.523934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:03.589970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7888f7-6612-4137-a6e8-ad259faf7ca3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:03.614074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:03.615282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:03.620315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:03.641290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:03.654347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:03.689898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917f4c9d694d40e7ac87addc9dabcc48'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.715863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917f4c9d694d40e7ac87addc9dabcc48'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.793399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917f4c9d694d40e7ac87addc9dabcc48'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.825035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917f4c9d694d40e7ac87addc9dabcc48'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.833199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917f4c9d694d40e7ac87addc9dabcc48'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:03.846870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 869/1708] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:03 2022] DELETE /identity/v3/users/917f4c9d694d40e7ac87addc9dabcc48 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:03.901795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:03.903816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d16e16-e6d3-42f5-87e8-015509fb396a service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:03.936315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 840/1709] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:52:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:52:04.336940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:04.336940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:04.588031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5b30-c574-4632-b5d7-cebda8dae2f1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:04.624342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:04.626683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:04.630313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:04.630867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] PATH_INFO: `/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:04.646162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27b8ec0878f4542a3902fe2a0f6619f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:04.649329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:04.763520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27b8ec0878f4542a3902fe2a0f6619f', 'request_id': 'req-052aa94d-184d-4a74-ad79-d6b6137e7093'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:04.775351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:04.787292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27b8ec0878f4542a3902fe2a0f6619f', 'request_id': 'req-052aa94d-184d-4a74-ad79-d6b6137e7093'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:04.801653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27b8ec0878f4542a3902fe2a0f6619f', 'request_id': 'req-052aa94d-184d-4a74-ad79-d6b6137e7093'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:04.829307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] The token cache is being invalidate because project d27b8ec0878f4542a3902fe2a0f6619f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:04.830385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:04.831634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-052aa94d-184d-4a74-ad79-d6b6137e7093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:04.834802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 870/1710] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:04 2022] DELETE /identity/v3/projects/d27b8ec0878f4542a3902fe2a0f6619f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:04.839722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:04.840698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:04.841369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:04.871709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:04.873624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:05.085805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 841/1711] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:05.092256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.092822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.094426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.117489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:05.137043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:05.206616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2f7ba-f03c-4e4b-9d23-80adea84de06 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:05.235830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:05.236512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.236899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.237119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] PATH_INFO: `/v3/users/83fd4adb347749a1ba1df6bf78e19118` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.246133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fd4adb347749a1ba1df6bf78e19118)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:05.248121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:05.308656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:05.321291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:05.374613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:05.376639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 871/1712] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:05.381269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:05.413960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a48e59-9a6b-42d3-b5f3-f29cb6c6889a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:05.439328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:05.442113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.442392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.442667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.443389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:05.443695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:05.445686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:05.475145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fd4adb347749a1ba1df6bf78e19118'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:05.481462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 872/1713] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:05.483005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:05.498131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fd4adb347749a1ba1df6bf78e19118'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:05.515766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e6d5e-4b6e-49db-926d-9af9f385a646 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:05.533369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:05.536058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.537581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.537581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.539908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fd4adb347749a1ba1df6bf78e19118'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:05.541610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:05.546887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:05.561717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fd4adb347749a1ba1df6bf78e19118'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:05.562677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e55c4ff5-d50f-409c-bc89-f43da42e450d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fd4adb347749a1ba1df6bf78e19118'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:05.591646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 842/1714] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:05 2022] DELETE /identity/v3/users/83fd4adb347749a1ba1df6bf78e19118 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:05.593523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.593850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.594102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.669515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 873/1715] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:05 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:05.687254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:05.711813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:05.716103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:05.725966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:05.948589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024788-7e80-4256-9247-5f9a6e519956 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:05.957737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 843/1716] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:05.960244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:05.961469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:05.962036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:05.965088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:05.966130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:05.968428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:06.036535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 874/1717] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:05 2022] POST /identity/v3/users => generated 284 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:06.041515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:06.069756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cdf7-7132-4338-9cf1-6fa912453d47 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:06.089534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:06.095507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:06.097086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:06.097378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:06.098085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:06.112075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:06.141232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:06.142201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 844/1718] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:06.147031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:06.147250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:06.183650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd379d-87f8-45ab-bab3-35a86a17344b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:06.200550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:06.202146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:06.203505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:06.203809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:06.205895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:06.214777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:06.251007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 845/1719] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:06.272089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:06.322727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66996555-9d99-41d5-a72a-dccc600a2e9d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:06.335652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:06.336294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:06.336571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:06.339649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:06.340922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da733888-3ad3-4c7f-882e-18ef2251a8af admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:06.361731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:06.366122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:06.369331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:06.370217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] PATH_INFO: `/v3/projects/e05c35bd659a4d90bbea6148492e69ff/users/7d05a9418022489aa447b37144086316/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:06.375421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:06.378703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c99a8a-71f9-4731-91a5-ff52d77237b6 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:06.413951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 875/1720] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:06.430303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05c35bd659a4d90bbea6148492e69ff, user_id=7d05a9418022489aa447b37144086316, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:06.439589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:06.497164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 846/1721] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:06 2022] PUT /identity/v3/projects/e05c35bd659a4d90bbea6148492e69ff/users/7d05a9418022489aa447b37144086316/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:06.508076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:06.508302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:06.508693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:06.592004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] MFA Rules not processed for user `7d05a9418022489aa447b37144086316`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:06.594335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:06.669944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:06.852368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 876/1722] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:06.859202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:07.066243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:07.122154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90017-2135-4e37-928e-d043f38c3ac5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:07.144723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:07.145357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:07.146703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:07.148588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:07.179733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:07.185699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:07.311700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 847/1723] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:06 2022] POST /identity/v3/projects => generated 367 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:07.317164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:07.348256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8762037-162f-43ef-89b3-4488b7ffe75e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:07.499440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:07.503469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:07.504129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:07.505592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:07.508540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:07.511634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:07.564274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 877/1724] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:07 2022] POST /identity/v3/users => generated 290 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:07.569475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:07.634083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d18c-f197-4d82-9ce3-c45166bd25ac admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:07.649735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:07.650565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:07.650853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:07.654338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:07.670171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:07.670171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:07.686388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 848/1725] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:07.687624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:07.706380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4cbb0-1278-43ea-9e17-48d18885c11e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:07.729002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:07.729662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:07.729891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:07.730183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] PATH_INFO: `/v3/projects/fb62f214ceb74bf291118da4206edbac/users/54cc7cefa0b743d39b640d1ad08ce388/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:07.766313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb62f214ceb74bf291118da4206edbac, user_id=54cc7cefa0b743d39b640d1ad08ce388, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:07.776128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:07.822893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 878/1726] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:07 2022] PUT /identity/v3/projects/fb62f214ceb74bf291118da4206edbac/users/54cc7cefa0b743d39b640d1ad08ce388/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:07.824187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:07.847065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001a9fc-3daa-446c-9a88-8e6d8d17832b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:08.142027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:08.149058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:08.154492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:08.154492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:08.154492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:08.154492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2d3ea-f353-4ec7-93e6-e76b1af6d1c2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:08.209712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:08.211117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 849/1727] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:08.238003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03320021-ab67-4910-a2c2-085dbf82fcfd admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:08.253634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:08.257326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:08.262518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:08.263337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] PATH_INFO: `/v3/projects/fb62f214ceb74bf291118da4206edbac/users/54cc7cefa0b743d39b640d1ad08ce388/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:08.272423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb62f214ceb74bf291118da4206edbac, user_id=54cc7cefa0b743d39b640d1ad08ce388, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:08.287117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb9dd6-dcfe-457d-bf38-03c5065298a5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:08.376061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 879/1728] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:08 2022] PUT /identity/v3/projects/fb62f214ceb74bf291118da4206edbac/users/54cc7cefa0b743d39b640d1ad08ce388/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:08.388552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:08.388863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:08.389140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:08.445450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:08.445818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:08.446098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:08.537694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] MFA Rules not processed for user `54cc7cefa0b743d39b640d1ad08ce388`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:08.538966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:08.541254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:08.542469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:08.806613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 880/1729] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Sep 20 04:52:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:08.815667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:08.820709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:08.866758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 850/1730] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:08.880972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:08.883645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:08.964250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:09.000516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:09.035759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae09a-f698-4824-8b05-b4dc43f5b804 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.045322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:09.046256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.046752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.047247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.073201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:09.075146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:09.098285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 851/1731] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:09.101395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:09.107467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:09.136635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd435-1601-4cb6-923a-6c6f86d95931 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.144152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:09.145925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.146419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.146913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.155771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5cf1d-fc48-464a-b3fb-0a3585b2b6d6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.163776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:09.164813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.165097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.165925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] PATH_INFO: `/v3/projects/c2376275534c4c0bb4d279a625a1bdcd` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.166309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:09.168558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:09.169371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2376275534c4c0bb4d279a625a1bdcd)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:09.173844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:09.179176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 852/1732] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:09.329326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2376275534c4c0bb4d279a625a1bdcd', 'request_id': 'req-845e2a24-59d1-4d82-a2c7-8d054d1207c2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.332231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:09.351293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afdc0-08d9-4591-99fd-66c56504161a service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.364174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2376275534c4c0bb4d279a625a1bdcd', 'request_id': 'req-845e2a24-59d1-4d82-a2c7-8d054d1207c2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.385478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2376275534c4c0bb4d279a625a1bdcd', 'request_id': 'req-845e2a24-59d1-4d82-a2c7-8d054d1207c2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.434844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] The token cache is being invalidate because project c2376275534c4c0bb4d279a625a1bdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:09.437271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.438981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-845e2a24-59d1-4d82-a2c7-8d054d1207c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.445386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 881/1733] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:08 2022] DELETE /identity/v3/projects/c2376275534c4c0bb4d279a625a1bdcd => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:09.453541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.453840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.455252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.518299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:09.522472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:09.536742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:09.538698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.539699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.542010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.548272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:09.635266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 882/1734] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:09.636462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:09.638252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:09.722500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d11ae-2a30-4b5c-9873-db7d65bb3fd7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.742247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:09.747031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:09.751523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.751836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.752118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] PATH_INFO: `/v3/projects/eac402e0f66c43359e9eb030cd550c04` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.757316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac402e0f66c43359e9eb030cd550c04)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:09.761495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:09.762958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133d704-9c60-4767-90e8-cf361b2c9781 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:09.787424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 853/1735] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:09.789568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:09.790026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:09.790903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:09.823989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:09.826990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:09.850288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 854/1736] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:09.856176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:09.890293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d12f2-f05c-45e9-b6ab-0cc57110d484 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:09.922183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:09.937070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac402e0f66c43359e9eb030cd550c04', 'request_id': 'req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.953823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac402e0f66c43359e9eb030cd550c04', 'request_id': 'req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.968105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac402e0f66c43359e9eb030cd550c04', 'request_id': 'req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.991640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] The token cache is being invalidate because project eac402e0f66c43359e9eb030cd550c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:09.992321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:09.994149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e525fbb8-9bb2-4afe-a3c3-e6fddd28ffbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:10.000662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 883/1737] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:09 2022] DELETE /identity/v3/projects/eac402e0f66c43359e9eb030cd550c04 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:10.004166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cada30f-11b2-464d-a519-5340239c89db None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.004750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cada30f-11b2-464d-a519-5340239c89db None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.005774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cada30f-11b2-464d-a519-5340239c89db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.118087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.120351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.121884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.122122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.123001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:10.123291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.125026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3cada30f-11b2-464d-a519-5340239c89db None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:10.129177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea55f2c-8789-4b49-93c9-36cac375913e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.129839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3cada30f-11b2-464d-a519-5340239c89db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:10.168561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 855/1738] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:10.168561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.169684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.169981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.181863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 884/1739] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:10.181863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cada30f-11b2-464d-a519-5340239c89db None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.182544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3cada30f-11b2-464d-a519-5340239c89db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:10.224866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cada30f-11b2-464d-a519-5340239c89db None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.234700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] MFA Rules not processed for user `c93f22104a7b4a7caf0cc8b7180ffc55`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:10.238193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:10.243604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.244300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.244650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.248431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.249259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.251165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f97000-e8db-4520-bd01-3f9f2ecdb5f5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.256139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:10.371045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 885/1740] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/projects => generated 355 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:10.371045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.371045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.371045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.403061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:10.404364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:10.519753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 856/1741] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:10.523169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.553837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466903f5-3916-4b4b-a600-90c0c0023426 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.644059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 886/1742] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:10.648764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.665923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc76e8b-d7f7-44b5-b8b1-e8d1edb2a044 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.681417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.684163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.684872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.685925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.687113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.687733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:10.688882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.689574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.693993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.694494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.695400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.696273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.700071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.765049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 887/1743] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:10.770047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.785868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a9ab-e19f-4a7b-95b3-11a9e3f8ec10 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.792127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 857/1744] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/users => generated 278 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:10.792957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.827545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.830546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.832349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.833261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ccdff-4807-4e3b-b890-da5f4d92dfdd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.833866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.834973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.852416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.872961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.873605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.878524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.878524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.881795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.887627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.929678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 858/1745] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:10.931835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:10.947827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3deca-a2c3-4f4d-8b90-b247c8a866a8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:10.955212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:10.956094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:10.956512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:10.957141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:10.958134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:10.960247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:10.966876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 888/1746] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:10.971116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.006279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 859/1747] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:11.008220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.035683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660422b-e2d7-47da-b0fd-4d1654d686b5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.265188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fad30-096a-466e-ab97-ec955eb748b5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.310359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:11.316965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.324160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.324507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.325230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:11.340578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:11.367988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:11.371364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.383651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.384568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] PATH_INFO: `/v3/projects/b81305bdc7934727b5c8e03390dc1f12/users/e75494b8fc134fb19149387c947aa9e1/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.465043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 889/1748] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:10 2022] POST /identity/v3/users => generated 293 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:11.469376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.473230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81305bdc7934727b5c8e03390dc1f12, user_id=e75494b8fc134fb19149387c947aa9e1, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:11.480553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84678f-4454-4582-97a2-17cbcee7fecb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:11.491301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:11.502621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8505-44a6-40a9-95c1-7cf251d7e95b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.523966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:11.525200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.525553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.525850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.526733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:11.528403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:11.567059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 890/1749] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:11.568425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.571399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 860/1750] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:11 2022] PUT /identity/v3/projects/b81305bdc7934727b5c8e03390dc1f12/users/e75494b8fc134fb19149387c947aa9e1/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:11.583470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.584003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.584424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.600107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339be75-07a3-49a3-9945-4444136085ac admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.632325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:11.701017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] MFA Rules not processed for user `e75494b8fc134fb19149387c947aa9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:11.704521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:11.762096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:11.762735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.763020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.763296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.763954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:11.766128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:11.793058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 891/1751] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:11.797362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.828073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab4b50-1250-4dc0-8580-418d99794408 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.835331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:11.836531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.836778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.838046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] PATH_INFO: `/v3/projects/eb3c5eada711464bb8c9d7f27dfcc742/users/7a61647410304a70a886595a08351097/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.868136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 861/1752] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:11.869627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3c5eada711464bb8c9d7f27dfcc742, user_id=7a61647410304a70a886595a08351097, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:11.873638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:11.877713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2af56-77ec-4f93-b105-1c79454deb16 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:11.910306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efdd7f-cbf7-4324-985a-d107520312ae None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:11.928721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 892/1753] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:11 2022] PUT /identity/v3/projects/eb3c5eada711464bb8c9d7f27dfcc742/users/7a61647410304a70a886595a08351097/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:11.934255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:11.934540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:11.934821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:11.987471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] MFA Rules not processed for user `7a61647410304a70a886595a08351097`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:11.988909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:12.112207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 893/1754] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:12.113652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.146847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.147491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.147780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.148054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.155580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.167295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.278483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 862/1755] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:11 2022] POST /identity/v3/projects => generated 355 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:12.279071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:12.284691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.299119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e77f3e-4e7b-4eea-acb1-bbf16af98207 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:12.449907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.454038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.454906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.455985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.457953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.465370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.528061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 894/1756] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:12 2022] POST /identity/v3/projects => generated 385 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:12.534391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90001c57-eaf6-491a-81b5-86ae7ffda71b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:12.536150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.585980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:12.696843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.698121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.698412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.698840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.699345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.710180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.781244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 863/1757] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:12 2022] POST /identity/v3/users => generated 284 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:12.785750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80761e5-43ff-458f-94d1-c6aa12b9b79e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:12.785923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.805433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.806796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.807073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.807548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.807985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.815755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.828117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce5f44-759f-4003-a06b-f934da24f031 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:12.843987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.850159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.850763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.851594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.853475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.855686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.880657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 895/1758] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:12 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:12.884572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.910669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 864/1759] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:12.911790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:12.919374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8c2e5-9de5-4593-9067-9687e5d96a2b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:12.942866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:12.944818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:12.945208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:12.947660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:12.950181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:12.968644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:12.977274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbb7c2-08cd-4f54-9746-e7e837bcdda8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.001872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 896/1760] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:13.002769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:13.010515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:13.011316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.011705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.011988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] PATH_INFO: `/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14/users/a6ba8260f1684411a962d842eb5c9444/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.039056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4fdf-76ca-4674-817a-cce5e3c5d329 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.056926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:13.057606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.057882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.058151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.058817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:13.060483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:13.097795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 897/1761] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:13.100326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:13.106315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4b0daaaee64f44b4a9562cff7eee14, user_id=a6ba8260f1684411a962d842eb5c9444, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:13.116862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:13.139075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15680a15-30b6-4289-88dc-6351ad06dccc admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.148573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 865/1762] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:12 2022] PUT /identity/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14/users/a6ba8260f1684411a962d842eb5c9444/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:13.153557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:13.288469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:13.319921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:13.321078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.322225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.325144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] PATH_INFO: `/v3/projects/2a9eba7f887c4da39b75d9bd26aece7e/users/2e7ef6cd72864b87a4dfddcbb87b2987/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.399115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9eba7f887c4da39b75d9bd26aece7e, user_id=2e7ef6cd72864b87a4dfddcbb87b2987, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:13.405223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b3f4e-0d82-489c-83f6-e09bc48f6e03 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.406657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54fcae-1e35-4abf-8ab4-d2bab7332254 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:13.440758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:13.442746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 898/1763] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:13 2022] PUT /identity/v3/projects/2a9eba7f887c4da39b75d9bd26aece7e/users/2e7ef6cd72864b87a4dfddcbb87b2987/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:13.454489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.455234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.456066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.462548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.462832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.467319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.471205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:13.478625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:13.503082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] MFA Rules not processed for user `2e7ef6cd72864b87a4dfddcbb87b2987`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:13.504961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:13.539706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 866/1764] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:13.553952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:13.594950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad0466-2832-4f2b-bf1c-ef7178496309 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.729621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:13.730883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.731073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.731350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] PATH_INFO: `/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14/users/a6ba8260f1684411a962d842eb5c9444/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.736570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4b0daaaee64f44b4a9562cff7eee14, user_id=a6ba8260f1684411a962d842eb5c9444, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:13.745155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1a392-aa17-49b3-b303-51716f90d747 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:13.767441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 899/1765] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:13.770480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:13.786383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 867/1766] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:13 2022] PUT /identity/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14/users/a6ba8260f1684411a962d842eb5c9444/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:13.798954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af800-4f61-4edd-aa8d-511a6cf2c88a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:13.799416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af800-4f61-4edd-aa8d-511a6cf2c88a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:13.799935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7af800-4f61-4edd-aa8d-511a6cf2c88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:13.818808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3ca2-bbf8-42be-ae0e-d4dbd338205e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:13.886505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:13.912231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7f7af800-4f61-4edd-aa8d-511a6cf2c88a None None] MFA Rules not processed for user `a6ba8260f1684411a962d842eb5c9444`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:13.915972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7f7af800-4f61-4edd-aa8d-511a6cf2c88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:14.042235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.044469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.045083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.046588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.048929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.056569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9470f-f809-46a4-a855-98e2c6f2768d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:14.147771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 900/1767] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:13 2022] POST /identity/v3/projects => generated 387 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:14.156281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b945-61d7-42f1-8ff9-a1be47a103f1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.156841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b945-61d7-42f1-8ff9-a1be47a103f1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.160247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489b945-61d7-42f1-8ff9-a1be47a103f1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.167140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 901/1768] 10.222.0.27 () {54 vars in 912 bytes} [Tue Sep 20 04:52:14 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:52:14.170906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489b945-61d7-42f1-8ff9-a1be47a103f1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:14.199489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 868/1769] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:14.202265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.202560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.204121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.310399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:14.316232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:14.412406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:14.488402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489b945-61d7-42f1-8ff9-a1be47a103f1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:14.523414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.524053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.524462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.524810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.525810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.526695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 869/1770] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:52:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:14.527382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7990049-b68f-42aa-8f03-d624cb1e1e12 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:14.527705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:14.534740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:14.580121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 902/1771] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:14 2022] POST /identity/v3/users => generated 300 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:14.585722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b6d4c-ee43-4c04-89a1-8ce6c8d42018 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.586019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b6d4c-ee43-4c04-89a1-8ce6c8d42018 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.586310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b6d4c-ee43-4c04-89a1-8ce6c8d42018 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.589190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 903/1772] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:52:14.590859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1442-3621-4c62-a81f-1b01da4b1543 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.591187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1442-3621-4c62-a81f-1b01da4b1543 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.591702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed1442-3621-4c62-a81f-1b01da4b1543 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.593462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 904/1773] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:52:14.594477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed1442-3621-4c62-a81f-1b01da4b1543 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:14.620274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed1442-3621-4c62-a81f-1b01da4b1543 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:14.633501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.634707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.635042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.635372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.636588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.637906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:14.655740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 905/1774] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:14.657013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:14.659344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:14.675967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8f1df-c3ad-4c1f-867c-c8fa20b3a1fb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:14.679909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.681188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.681617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.682072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.691601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.693793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:14.706365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 906/1775] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:14.707736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:14.724010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf85d-0e17-4258-927f-bbd5e90714da service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:14.732111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.736049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.737895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.737895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] PATH_INFO: `/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46/users/883cc3693419453fb04945a76e84c4b9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.756117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d139d-36db-4ddb-8279-d5dfa5c9168e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:14.764004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:14.771754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.772129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.772458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.796519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f76b2cbae8f4cdb86b2de2596ab5c46, user_id=883cc3693419453fb04945a76e84c4b9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.809226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aca625-9408-48f0-857a-186ba1790bc6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:14.848840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 907/1776] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:14 2022] PUT /identity/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46/users/883cc3693419453fb04945a76e84c4b9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:14.854135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:14.854425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:14.854700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:14.913946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:14.915774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:14.964163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:14.968136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:14.969975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7833903-1380-4118-a06e-c410a8506fa5 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:15.016919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 870/1777] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:15.024069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:15.025060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:15.032814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:15.128867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:15.143103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:15.194729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 908/1778] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:52:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:15.199217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:15.403670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 871/1779] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:52:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:15.405504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:15.420913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:15.452186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748d3fe-41d7-4845-9786-2d4df2b338c7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:15.462973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:15.465229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689f3e9-9afa-4555-bc51-f052a6dcbb38 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:15.466466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:15.466748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:15.467025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:15.480476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:15.482307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:15.495751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:15.497301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:15.497595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:15.497874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] PATH_INFO: `/v3/users/5a9cce1e994949d78af6066042b45869` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:15.505781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9cce1e994949d78af6066042b45869)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:15.513955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:15.522973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 909/1780] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:15.538628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:15.548276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:15.679709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:15.750424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9cce1e994949d78af6066042b45869'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:15.788444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9cce1e994949d78af6066042b45869'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:15.829970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca78aab-20ae-4c85-af2a-9613a3171d48 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:15.844391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:15.845062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:15.845343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:15.845624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:15.847434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9cce1e994949d78af6066042b45869'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:15.881685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9cce1e994949d78af6066042b45869'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:15.882490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9cce1e994949d78af6066042b45869'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:15.906855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 872/1781] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:15 2022] DELETE /identity/v3/users/5a9cce1e994949d78af6066042b45869 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:15.922822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:15.927770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.098969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:16.100937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e6d2a-21bf-48ec-9e7b-ec19d2cefccb service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:16.117707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b14f21-71f3-4ec1-bfce-31999980fcc8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:16.126160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:16.127376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:16.136516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 910/1782] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:16.140498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:16.140812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:16.141093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:16.148760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:16.160538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] PATH_INFO: `/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46/users/883cc3693419453fb04945a76e84c4b9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:16.168716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f76b2cbae8f4cdb86b2de2596ab5c46, user_id=883cc3693419453fb04945a76e84c4b9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:16.175996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1dcebf-ca1b-4b32-9e6f-6461085d8838 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:16.184050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:16.197713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:16.199002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.229463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 873/1783] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:15 2022] PUT /identity/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46/users/883cc3693419453fb04945a76e84c4b9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:16.234708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:16.235015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:16.235467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:16.329306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] MFA Rules not processed for user `883cc3693419453fb04945a76e84c4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:16.330568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.413003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 911/1784] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:16.420550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:16.424435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.718773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 874/1785] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:16.731270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:16.736253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.755419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c9eb1-b410-46d2-a04b-cbe0c67f152f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:16.763664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:16.764911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:16.765385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:16.765906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:16.800821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:16.805093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:16.832434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 912/1786] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:16.833536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:16.842728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:16.911790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:16.976157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104a8bf-604d-4b9b-87b9-5d6c4d0bfd46 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:16.997253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:16.998544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:16.998831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:16.999106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:17.004543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:17.044535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67226cc4-e859-4b2b-9e97-0c82741d03b8 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:17.046861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:17.052634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:17.077481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:17.079608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:17.080097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:17.080763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 875/1787] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:17.096588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:17.139647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:17.154174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:17.195791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 913/1788] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:17.332303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:17.337789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:17.397512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2c9b-3b20-4f75-9310-d148ba9c79c2 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:17.421061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:17.421787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:17.422070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:17.422500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:17.437789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:17.469145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:17.470701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe077d8-345b-450f-98b7-5226e3b7fff1 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:17.522412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 876/1789] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:19.034374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.038218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:19.082148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d90b-29a7-4a99-9dee-7bc018ba5296 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:19.096781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:19.098536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.099690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.100236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] PATH_INFO: `/v3/projects/219fc1b69bb44df3af1e0f669f0c423b` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.103548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] RBAC: Authorizing `identity:delete_project(project_id=219fc1b69bb44df3af1e0f669f0c423b)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:19.122157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:19.285048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219fc1b69bb44df3af1e0f669f0c423b', 'request_id': 'req-4a3c31cd-559e-4e5e-9401-07ff0076a34e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:19.312567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219fc1b69bb44df3af1e0f669f0c423b', 'request_id': 'req-4a3c31cd-559e-4e5e-9401-07ff0076a34e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:19.343224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219fc1b69bb44df3af1e0f669f0c423b', 'request_id': 'req-4a3c31cd-559e-4e5e-9401-07ff0076a34e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:19.378286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] The token cache is being invalidate because project 219fc1b69bb44df3af1e0f669f0c423b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:19.378286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:19.380160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4a3c31cd-559e-4e5e-9401-07ff0076a34e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:19.384635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 914/1790] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:19 2022] DELETE /identity/v3/projects/219fc1b69bb44df3af1e0f669f0c423b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:19.391937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.393261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.393821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.451543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:19.453647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:19.551401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:19.611122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 877/1791] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:19.612111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.612400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.612725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.671574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:19.672854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:19.720845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 915/1792] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:19.722333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.741115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cca295-8196-4bf9-8ecb-ea12c18d44b7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:19.747378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:19.748119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.748493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.748907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.749783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:19.750301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:19.752021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:19.767169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 878/1793] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:52:19.772028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.784350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b0e32-4e04-471b-b29b-76aab59579ad None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:19.791369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:19.792562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.793191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.804957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.806476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:19.808869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:19.818168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.820117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:19.859103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 916/1794] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:19 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:19.860719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.926265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9471-88c4-4c34-a05d-c5210d08f386 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:19.951506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c777-764a-47dc-a563-db962d578527 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:19.958026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:19.958787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.959063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.959339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.960085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:19.962666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:19.983519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:19.984353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:19.984777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:19.985590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:19.986390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 917/1795] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:19 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:19.987395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:19.989998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:19.991710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:20.002758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:20.007057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5404014-e45f-4944-82b9-90b0ee5972da admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.014118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.015285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.015599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.016627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] PATH_INFO: `/v3/users/7d05a9418022489aa447b37144086316` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.018578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d05a9418022489aa447b37144086316)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.021045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.072959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.074426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.085829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 879/1796] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:20.087055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.102460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739c277-29fb-46fc-a21f-f4418223702c service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.106506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d05a9418022489aa447b37144086316'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:20.109043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.109817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.110303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.110651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.112838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.114676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.115864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d05a9418022489aa447b37144086316'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:20.132611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d05a9418022489aa447b37144086316'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:20.139910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 880/1797] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:20.141398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.145144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d05a9418022489aa447b37144086316'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:20.146558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-306edaab-e2e5-4847-97ed-fda41ade27cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d05a9418022489aa447b37144086316'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:20.174310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 918/1798] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:19 2022] DELETE /identity/v3/users/7d05a9418022489aa447b37144086316 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:20.176026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.176390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.176764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.177683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130bc8b-4caf-40f9-bcc1-4e0f7caedc19 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.200716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:20.203892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:20.253268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:20.294364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 919/1799] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:20.304758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.307079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:20.307219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.307708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.307987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.308263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.308947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.310565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.329393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 881/1800] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:20.333254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.335835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:20.364940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61df17-a19e-4d1a-8c37-e20c848ae4ce admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.374038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.374968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.375232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.375507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] PATH_INFO: `/v3/projects/41dea1ef77ba41bb98681a9ac02d6f80/users/b62fbf70ee87478b9190d2ababcb0773/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.395250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dea1ef77ba41bb98681a9ac02d6f80, user_id=b62fbf70ee87478b9190d2ababcb0773, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.398863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769ccb2-0130-4e6c-be6e-625c38ac2db2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.417958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 882/1801] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:20 2022] PUT /identity/v3/projects/41dea1ef77ba41bb98681a9ac02d6f80/users/b62fbf70ee87478b9190d2ababcb0773/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:20.429190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.429616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.429964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.433377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e1642-d09b-4691-a447-e405df3e8543 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.490647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] MFA Rules not processed for user `b62fbf70ee87478b9190d2ababcb0773`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:20.490647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:20.505423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:20.553419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.554053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.554365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.554644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.641379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 883/1802] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:20.649464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.657391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.659276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.670679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974cedf-ee0c-4b6e-9895-3138240d80ed None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.671229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 920/1803] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:20.677445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.678236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.678508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.678787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.679471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.682045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.721116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 884/1804] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:20 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:20.725401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.735826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a444ea-0be4-482a-8c60-a512d95ccd46 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.810949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.811579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.811862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.812137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.812930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.814944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.855823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 921/1805] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:20 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:20.856670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.867061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d866f-a3f2-4c8a-887c-76e6fa6c5ea5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.873316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.873950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.874261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.874541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.875579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.876929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.895093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 885/1806] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:20.903903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.916840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb995d2-9d87-4551-b786-da273fc77a47 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:20.922214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:20.923257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:20.923257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:20.923521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] PATH_INFO: `/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b/users/ee0759c3c42b4c24974cee9da73433a8/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:20.942329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1ec00f60ad4ece89db3c38ad007d2b, user_id=ee0759c3c42b4c24974cee9da73433a8, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:20.945133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:20.962369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 922/1807] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:20 2022] PUT /identity/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b/users/ee0759c3c42b4c24974cee9da73433a8/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:20.966432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:20.981446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5820e-7ea8-45d1-902e-8569c1120f79 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:21.077607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:21.078130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.078415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.078751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.079339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:21.080983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c99795-2e58-48d3-bc1c-b95334e30aa9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:21.097179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:21.097995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 886/1808] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:21.111925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e42e21-bdd8-43de-91f9-4a082fba0044 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:21.119891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:21.120664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.121327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.121756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] PATH_INFO: `/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b/users/ee0759c3c42b4c24974cee9da73433a8/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.126977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1ec00f60ad4ece89db3c38ad007d2b, user_id=ee0759c3c42b4c24974cee9da73433a8, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:21.131724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:21.158021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 923/1809] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:21 2022] PUT /identity/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b/users/ee0759c3c42b4c24974cee9da73433a8/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:21.159669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.159956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.160238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.168206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:21.203811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] MFA Rules not processed for user `ee0759c3c42b4c24974cee9da73433a8`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:21.205141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:21.325788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 887/1810] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:21.342870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:21.344768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:21.370626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:21.372315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:21.409027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:21.452416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e6844-9de2-4f45-81f7-4ac7a155ed08 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:21.457427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:21.458490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.459011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.459571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.468886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e9bbd-e0cc-43cf-9609-a3f006ceab13 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:21.478884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:21.479648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.480051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.480389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] PATH_INFO: `/v3/projects/e05c35bd659a4d90bbea6148492e69ff` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.481650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05c35bd659a4d90bbea6148492e69ff)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:21.484532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:21.512965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:21.514739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:21.529624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 924/1811] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:21.547461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05c35bd659a4d90bbea6148492e69ff', 'request_id': 'req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.554259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05c35bd659a4d90bbea6148492e69ff', 'request_id': 'req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.565482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05c35bd659a4d90bbea6148492e69ff', 'request_id': 'req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.596202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] The token cache is being invalidate because project e05c35bd659a4d90bbea6148492e69ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:21.596758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.597901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.602964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 888/1812] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:21 2022] DELETE /identity/v3/projects/e05c35bd659a4d90bbea6148492e69ff => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:21.604164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:21.620770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6e140-42b0-4501-b647-6b0379da59a9 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:21.747356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:21.748112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:21.748807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:21.749156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] PATH_INFO: `/v3/users/54cc7cefa0b743d39b640d1ad08ce388` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:21.750680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cc7cefa0b743d39b640d1ad08ce388)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:21.752774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:21.812328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cc7cefa0b743d39b640d1ad08ce388'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.824981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cc7cefa0b743d39b640d1ad08ce388'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.842901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cc7cefa0b743d39b640d1ad08ce388'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.856563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cc7cefa0b743d39b640d1ad08ce388'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.857442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9eda6b12-86c3-4a99-871f-19386b764154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cc7cefa0b743d39b640d1ad08ce388'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:21.864283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 925/1813] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:21 2022] DELETE /identity/v3/users/54cc7cefa0b743d39b640d1ad08ce388 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:22.229496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.231009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:22.244683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f38b-b9dc-47b4-9776-dd667d92ce63 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.268153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:22.320677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.321842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.322123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.322393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] PATH_INFO: `/v3/projects/fb62f214ceb74bf291118da4206edbac` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.323514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb62f214ceb74bf291118da4206edbac)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.325463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.386400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb62f214ceb74bf291118da4206edbac', 'request_id': 'req-423865f6-727e-4ea6-885f-3c1e1f218ad6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:22.391982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb62f214ceb74bf291118da4206edbac', 'request_id': 'req-423865f6-727e-4ea6-885f-3c1e1f218ad6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:22.401503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb62f214ceb74bf291118da4206edbac', 'request_id': 'req-423865f6-727e-4ea6-885f-3c1e1f218ad6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:22.418624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] The token cache is being invalidate because project fb62f214ceb74bf291118da4206edbac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:22.418914 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:22.419557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-423865f6-727e-4ea6-885f-3c1e1f218ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:22.421095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 889/1814] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:22 2022] DELETE /identity/v3/projects/fb62f214ceb74bf291118da4206edbac => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:22.425922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.426408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.426592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.446441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:22.447691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:22.500727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:22.517576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 926/1815] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:22.526024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.527974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.527974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.557315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:22.558969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:22.572020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 890/1816] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:22.572363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.584896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fd85-a305-46b9-9479-db4607d57107 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.591089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.591684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.591961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.592232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.592920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:22.593223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.594821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.607882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 927/1817] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:22.612138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.622556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456b8bc-0bf9-4abd-9f62-e75bb3ccd035 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.629399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.630181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.630461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.630734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.631382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.633535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.687358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 891/1818] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:22 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:22.689532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.701366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d395b5-39d9-400c-a706-3e556b163772 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.772434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.773560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.773560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.774557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.775001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.777299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.804943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 928/1819] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:22 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:22.813685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.826587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf6b76-3987-44a7-aa13-86daa937365e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.831501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.832108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.832392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.832701 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.833345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.834992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.845366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 892/1820] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:22.849098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.859588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccde3cd-08f9-4fd4-be96-469ed5192cca admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.864076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.864723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.865013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.865280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.865913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.867502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b99ab-5d6e-4dfb-9629-b62e26eea4a6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.880744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 929/1821] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:22.885722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:22.895839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935002-ed56-4288-9475-bb20af67f95b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:22.900807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:22.901471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.901874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.902157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] PATH_INFO: `/v3/projects/c0e05d458fb74e3b9e0bcd9a3e662a0f/users/4f824db60faf4f959158f112c528b463/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.921978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e05d458fb74e3b9e0bcd9a3e662a0f, user_id=4f824db60faf4f959158f112c528b463, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:22.925469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:22.940555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 893/1822] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:22 2022] PUT /identity/v3/projects/c0e05d458fb74e3b9e0bcd9a3e662a0f/users/4f824db60faf4f959158f112c528b463/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:22.950898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:22.951189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:22.951496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:22.983125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] MFA Rules not processed for user `4f824db60faf4f959158f112c528b463`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:22.984370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.064427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 930/1823] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:23.077545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:23.079395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.139082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:23.159418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:23.160900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.179027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32602295-4c2e-4dfc-81ab-69dc38d5532f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:23.183783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:23.184595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:23.184750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:23.184964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:23.194031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:23.195880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fd8ee-6504-4aa3-9414-9ddcd626917f service glance] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:23.206175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 894/1824] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:23.219859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:23.235220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0e31-8700-4a4f-8a63-3eddd0cac085 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:23.239878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:23.240512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:23.240857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:23.241110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] PATH_INFO: `/v3/users/b62fbf70ee87478b9190d2ababcb0773` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:23.242236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b62fbf70ee87478b9190d2ababcb0773)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:23.244722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:23.294004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b62fbf70ee87478b9190d2ababcb0773'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.301064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b62fbf70ee87478b9190d2ababcb0773'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.314109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b62fbf70ee87478b9190d2ababcb0773'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.322429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b62fbf70ee87478b9190d2ababcb0773'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.323213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b62fbf70ee87478b9190d2ababcb0773'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.329884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 931/1825] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:23 2022] DELETE /identity/v3/users/b62fbf70ee87478b9190d2ababcb0773 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:23.334282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:23.334574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:23.334906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:23.356109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:23.357265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.426916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 895/1826] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:23.436737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:23.438154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.496192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791b462-c8a2-4a07-bd5a-31271e2413a9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:23.499944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:23.500549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:23.500860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:23.501115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:23.510829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:23.512293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:23.522221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 932/1827] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:23.810281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:23.812142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:23.839439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94231cd-e3d1-4e0b-ae4c-d06e73313a8d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:23.847746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:23.848480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:23.848800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:23.849171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] PATH_INFO: `/v3/projects/41dea1ef77ba41bb98681a9ac02d6f80` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:23.850326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dea1ef77ba41bb98681a9ac02d6f80)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:23.853361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:23.906765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dea1ef77ba41bb98681a9ac02d6f80', 'request_id': 'req-727edb0b-261f-496b-aab1-d86bcb368130'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.912303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dea1ef77ba41bb98681a9ac02d6f80', 'request_id': 'req-727edb0b-261f-496b-aab1-d86bcb368130'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.920688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dea1ef77ba41bb98681a9ac02d6f80', 'request_id': 'req-727edb0b-261f-496b-aab1-d86bcb368130'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.932702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] The token cache is being invalidate because project 41dea1ef77ba41bb98681a9ac02d6f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:23.932994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.933636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:23.938177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 896/1828] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:23 2022] DELETE /identity/v3/projects/41dea1ef77ba41bb98681a9ac02d6f80 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:23.939160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:23.948849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657517d8-af9b-4fe3-abb8-24a76aead3ea service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.006161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.006788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.007065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.007335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] PATH_INFO: `/v3/users/ee0759c3c42b4c24974cee9da73433a8` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.010111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0759c3c42b4c24974cee9da73433a8)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.012041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.029842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:24.077607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0759c3c42b4c24974cee9da73433a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.089988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0759c3c42b4c24974cee9da73433a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.108129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0759c3c42b4c24974cee9da73433a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.115747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0759c3c42b4c24974cee9da73433a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.116403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a21c4c11-f4b8-47c8-b57d-e2ea580d26ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0759c3c42b4c24974cee9da73433a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.122538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 933/1829] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:23 2022] DELETE /identity/v3/users/ee0759c3c42b4c24974cee9da73433a8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:24.425507 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.426924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:24.442376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727edb0b-261f-496b-aab1-d86bcb368130 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.490526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:24.502860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.503622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.503899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.504170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] PATH_INFO: `/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.505328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1ec00f60ad4ece89db3c38ad007d2b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.507211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.552773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1ec00f60ad4ece89db3c38ad007d2b', 'request_id': 'req-bf437ffe-2e77-4d93-837b-057653b1e25f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.559574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1ec00f60ad4ece89db3c38ad007d2b', 'request_id': 'req-bf437ffe-2e77-4d93-837b-057653b1e25f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.567521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1ec00f60ad4ece89db3c38ad007d2b', 'request_id': 'req-bf437ffe-2e77-4d93-837b-057653b1e25f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.581029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] The token cache is being invalidate because project 3d1ec00f60ad4ece89db3c38ad007d2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:24.581349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.582005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf437ffe-2e77-4d93-837b-057653b1e25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:24.583532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 897/1830] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:24 2022] DELETE /identity/v3/projects/3d1ec00f60ad4ece89db3c38ad007d2b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:24.589232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.589716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.589792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.613262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:24.614578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:24.681200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 934/1831] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:24.688823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.689111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.689390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.712570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:24.712570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:24.720650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 898/1832] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:24.725075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.735212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf844962-9a88-410e-bfa6-693274bda86d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.739936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.741118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.741118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.741118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.741784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:24.742120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.744482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.767793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 935/1833] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:24.772472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.783440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ccac-b7f7-41d8-8f9d-debceb3c673c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.788409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.789276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.790119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.790441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.791311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.793665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.823933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 899/1834] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:24 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:24.825231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.846466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d7564-f5cd-4fd8-978a-3c4aa2a925f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.913935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.915182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.915493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.915763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.916396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.918400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.947006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 936/1835] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:24 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:24.951066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.961933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08b3c8-6f7a-4ee4-8d73-8ec4fb607ccd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:24.966441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:24.967030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:24.967306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:24.967577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:24.968212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:24.969871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:24.981049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 900/1836] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:24.985077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:24.995374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a73486-6784-4f1d-81a8-0df1c0c19b48 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.000283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.000957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.001245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.001518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.002201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.004264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419589b-1bb1-43fc-ab64-8b853670d3d1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.011579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:25.019544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 937/1837] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:25.023560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.034370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9efd6-de94-4fa3-9d9d-575f95a221eb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.038932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.040209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.040489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.040796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] PATH_INFO: `/v3/projects/d3938f97b00745008abd7cd4217666ec/users/d17f5a02b60c4b34b159b50db3373fde/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.055160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3938f97b00745008abd7cd4217666ec, user_id=d17f5a02b60c4b34b159b50db3373fde, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.057992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.076734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 901/1838] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:25 2022] PUT /identity/v3/projects/d3938f97b00745008abd7cd4217666ec/users/d17f5a02b60c4b34b159b50db3373fde/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:25.095354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.095640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.095913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.148351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] MFA Rules not processed for user `d17f5a02b60c4b34b159b50db3373fde`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:25.151131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:25.244634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 938/1839] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:25.249083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.324083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5b0de-3752-4408-98d8-1a96a33a28fb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.328897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.329543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.329778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.330049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.330903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.333076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.369090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 902/1840] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:25.370458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.381244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7082452-82ae-4c00-934b-cce357f568d7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.452456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.453193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.453541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.453876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.454575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.456526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.478706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 939/1841] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:25 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:25.482645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.488173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:25.494286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ff8e8-b12e-4c8c-be5e-a195f86999b7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.499590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.500184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.500480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.500768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.501403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.502990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.513779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 903/1842] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:25.517803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.528462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26229088-c1b8-4252-90b2-e54cb4b374fb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.533010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.533646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.533925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.534196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] PATH_INFO: `/v3/projects/0ef24df1d3e7449f870a99355d7965c8/users/e9803a3b2c8549e1a04bb6ac0ef7b4af/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.548491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef24df1d3e7449f870a99355d7965c8, user_id=e9803a3b2c8549e1a04bb6ac0ef7b4af, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.551165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.564553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 940/1843] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:25 2022] PUT /identity/v3/projects/0ef24df1d3e7449f870a99355d7965c8/users/e9803a3b2c8549e1a04bb6ac0ef7b4af/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:25.568416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.579523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2e815-15ff-4bec-9b55-d962a7e8bcd4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.634150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.634749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.635032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.635307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.635946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.637583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb75889-eb31-461c-bcca-0da7ceb7aa63 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.648327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 904/1844] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:25.653278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.667815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82071c-1d86-4449-be87-d4a2b8e4b03f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.672920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.673598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.673880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.674152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] PATH_INFO: `/v3/projects/0ef24df1d3e7449f870a99355d7965c8/users/e9803a3b2c8549e1a04bb6ac0ef7b4af/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.676090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef24df1d3e7449f870a99355d7965c8, user_id=e9803a3b2c8549e1a04bb6ac0ef7b4af, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.678798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.689909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 941/1845] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:25 2022] PUT /identity/v3/projects/0ef24df1d3e7449f870a99355d7965c8/users/e9803a3b2c8549e1a04bb6ac0ef7b4af/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:25.701017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3915d-68fe-4a3a-99c0-019c71ad3e36 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.701314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3915d-68fe-4a3a-99c0-019c71ad3e36 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.701581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3915d-68fe-4a3a-99c0-019c71ad3e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.729264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7ec3915d-68fe-4a3a-99c0-019c71ad3e36 None None] MFA Rules not processed for user `e9803a3b2c8549e1a04bb6ac0ef7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:25.730460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7ec3915d-68fe-4a3a-99c0-019c71ad3e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:25.799277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 905/1846] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:25.814539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:25.816268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:25.862645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:25.875469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86691708-b8a5-42d5-9511-59dfe9a26eae admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:25.879701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:25.880317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:25.880614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:25.880906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:25.944578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:25.946673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c9c75-70e9-4740-8285-f351ea9ea172 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:25.956142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 942/1847] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:32.115167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:32.115648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:32.115755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:32.117083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 906/1848] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:52:32.143691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:32.143691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:32.143691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:32.169181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:32.170506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:32.246401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 943/1849] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:52:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:32.262887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:32.264593 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:32.287959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369569a-bc41-4612-902e-12a681d7c589 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:32.293958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:32.294600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:32.294880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:32.295152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:32.305657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:32.307321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:32.318956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 907/1850] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:32.781166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:32.792007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952e210-7d3c-437c-bb0f-79e9feab592f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:32.796967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:32.798169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:32.799285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:32.799285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:32.817593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:32.833941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:32.847130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 944/1851] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:33.319429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:33.321882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:33.339637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651557a-64a8-49b8-a805-cae9d96b3473 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:33.345893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:33.346823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:33.347177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:33.347520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:33.349504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:33.438854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:33.440922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef94e5cb-560b-45dd-a7ea-2fb4edc098d0 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:33.445209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:33.456361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 908/1852] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:39.196463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:39.198632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:39.273997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143a22e-ed71-413f-94f6-795b4c8ed288 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:39.278491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:39.279116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:39.279393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:39.279665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] PATH_INFO: `/v3/users/7a61647410304a70a886595a08351097` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:39.280807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a61647410304a70a886595a08351097)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:39.282786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:39.299512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:39.343585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a61647410304a70a886595a08351097'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:39.353057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a61647410304a70a886595a08351097'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:39.366270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a61647410304a70a886595a08351097'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:39.374284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a61647410304a70a886595a08351097'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:39.375579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a61647410304a70a886595a08351097'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:39.383521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 945/1853] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:39 2022] DELETE /identity/v3/users/7a61647410304a70a886595a08351097 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:39.387878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:39.388319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:39.388430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:39.417180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:39.418401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:39.485456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 909/1854] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:39.496470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:39.498048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:39.566134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b007d-d6db-4a37-9a63-d880b9fe13ed admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:39.569704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:39.570835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:39.571140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:39.571432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:39.580221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:39.581670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:39.591680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 946/1855] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:39.873822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:39.875750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:39.896735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af412c-9088-440f-a862-5d51b8a729d6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:39.903413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:39.912151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:39.912570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:39.912871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] PATH_INFO: `/v3/projects/eb3c5eada711464bb8c9d7f27dfcc742` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:39.914802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3c5eada711464bb8c9d7f27dfcc742)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:39.916847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:40.059728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3c5eada711464bb8c9d7f27dfcc742', 'request_id': 'req-7b000a12-150a-4e85-a747-879a24b46225'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.070561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3c5eada711464bb8c9d7f27dfcc742', 'request_id': 'req-7b000a12-150a-4e85-a747-879a24b46225'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.095432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3c5eada711464bb8c9d7f27dfcc742', 'request_id': 'req-7b000a12-150a-4e85-a747-879a24b46225'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.109307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] The token cache is being invalidate because project eb3c5eada711464bb8c9d7f27dfcc742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:40.109751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.110492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.112645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 910/1856] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:39 2022] DELETE /identity/v3/projects/eb3c5eada711464bb8c9d7f27dfcc742 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:40.116635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:40.129986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0bfdd-6d7a-43f5-baa9-d60d7e144108 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:40.212526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:40.213184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:40.213462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:40.213734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] PATH_INFO: `/v3/users/2e7ef6cd72864b87a4dfddcbb87b2987` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:40.214842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7ef6cd72864b87a4dfddcbb87b2987)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:40.217174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:40.261849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:40.269479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7ef6cd72864b87a4dfddcbb87b2987'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.279265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7ef6cd72864b87a4dfddcbb87b2987'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.293733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7ef6cd72864b87a4dfddcbb87b2987'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.301608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7ef6cd72864b87a4dfddcbb87b2987'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.302452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7ef6cd72864b87a4dfddcbb87b2987'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.311631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 947/1857] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:40 2022] DELETE /identity/v3/users/2e7ef6cd72864b87a4dfddcbb87b2987 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:40.596047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:40.598042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:40.632161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b000a12-150a-4e85-a747-879a24b46225 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:40.691433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:40.692215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:40.692499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:40.692801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] PATH_INFO: `/v3/projects/2a9eba7f887c4da39b75d9bd26aece7e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:40.693947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9eba7f887c4da39b75d9bd26aece7e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:40.695917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:40.721289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:40.744869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9eba7f887c4da39b75d9bd26aece7e', 'request_id': 'req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.750522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9eba7f887c4da39b75d9bd26aece7e', 'request_id': 'req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.758622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9eba7f887c4da39b75d9bd26aece7e', 'request_id': 'req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.769768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] The token cache is being invalidate because project 2a9eba7f887c4da39b75d9bd26aece7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:40.770057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.770936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.772464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 911/1858] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:40 2022] DELETE /identity/v3/projects/2a9eba7f887c4da39b75d9bd26aece7e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:40.776962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:40.778348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:40.790936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e5a3-3f13-495c-a555-63479459691a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:40.849914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:40.850542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:40.850820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:40.851089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] PATH_INFO: `/v3/users/883cc3693419453fb04945a76e84c4b9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:40.852175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883cc3693419453fb04945a76e84c4b9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:40.854104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:40.901917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883cc3693419453fb04945a76e84c4b9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.908784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883cc3693419453fb04945a76e84c4b9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.920508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883cc3693419453fb04945a76e84c4b9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.928525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883cc3693419453fb04945a76e84c4b9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.929238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a5dc0661-7d20-4f7d-be73-e916bccfe0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883cc3693419453fb04945a76e84c4b9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:40.935794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 948/1859] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:40 2022] DELETE /identity/v3/users/883cc3693419453fb04945a76e84c4b9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:41.233418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.234781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:41.247550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f1c5b-3af0-4282-a1eb-336b53b191c1 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.317963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.318607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.319021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.319293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] PATH_INFO: `/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.321083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f76b2cbae8f4cdb86b2de2596ab5c46)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.323198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.379847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f76b2cbae8f4cdb86b2de2596ab5c46', 'request_id': 'req-9b1f4797-a00c-4a2a-80cd-22fac12fef27'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:41.385930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f76b2cbae8f4cdb86b2de2596ab5c46', 'request_id': 'req-9b1f4797-a00c-4a2a-80cd-22fac12fef27'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:41.393476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f76b2cbae8f4cdb86b2de2596ab5c46', 'request_id': 'req-9b1f4797-a00c-4a2a-80cd-22fac12fef27'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:41.406008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] The token cache is being invalidate because project 7f76b2cbae8f4cdb86b2de2596ab5c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:41.406300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:41.407035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9b1f4797-a00c-4a2a-80cd-22fac12fef27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:41.408464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 912/1860] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:41 2022] DELETE /identity/v3/projects/7f76b2cbae8f4cdb86b2de2596ab5c46 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:41.414534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.415005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.415113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.436091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:41.437310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:41.466020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:41.496332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 949/1861] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:41.504845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.505143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.505406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.525235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:41.526423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:41.534047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 913/1862] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:41.538034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.548146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eef677-a61f-4d36-bddd-73f71badf6d1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.552708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.553298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.553574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.553841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.554468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:41.554787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.556381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.567652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 950/1863] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:41.572919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.581866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30558545-a311-4b98-b2c3-782ec46507dc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.586921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.587505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.587782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.588049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.588705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.590582 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.618025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 914/1864] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:41.622263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.632022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8d385-f297-4c28-a0b1-a37ba014c5f5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.688754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.689364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.689645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.689918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.690554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.692486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.713664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 951/1865] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:41.717868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.728360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b4f9d-8d6e-425f-b4a6-3022037dfb6d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.732849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.733430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.733707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.733977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.734607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.736176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.746434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 915/1866] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:41.750108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.759777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7bbb2-4769-4e3f-9287-7f90aee89337 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.764379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.765010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.765293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.765565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.766207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.767818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125aaf9f-7ebc-4e41-97e2-b66d4a213f26 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.778891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 952/1867] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:41.782772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:41.787752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:41.794081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fbb84-3cbf-45bf-b07f-8f1ed8f8a740 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:41.799847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:41.800756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.801229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.801583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] PATH_INFO: `/v3/projects/297ad9e859224973ae21f156ddc39652/users/c5085765482148a79ddf0cca3c85bee5/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.816977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297ad9e859224973ae21f156ddc39652, user_id=c5085765482148a79ddf0cca3c85bee5, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:41.819598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:41.832228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 916/1868] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:41 2022] PUT /identity/v3/projects/297ad9e859224973ae21f156ddc39652/users/c5085765482148a79ddf0cca3c85bee5/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:41.843098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:41.843385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:41.843657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:41.872457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] MFA Rules not processed for user `c5085765482148a79ddf0cca3c85bee5`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:41.873701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:41.936356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 953/1869] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:41.940762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.001609 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa35dc-ef3d-41f9-a878-82a37ffb5172 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.006340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.006956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.007265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.007571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.008213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.010171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.036232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 917/1870] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:41 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:42.040857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.051418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f44f8-8150-4f6b-952b-9b7fca642b42 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.079929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:42.108989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.109591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.109867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.110141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.110763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.112747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.132920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 954/1871] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:42 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:42.136729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.147321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3e7fb-db05-4de2-a4ee-5014280679ff admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.152505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.153134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.153409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.153681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.154315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.155901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.167127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 918/1872] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:42.171114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.181145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da219-5ee9-4e10-b503-58baf38b2c64 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.185679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.186278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.186617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.186831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.187463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.189191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb83de-5c4f-40d1-9d51-62a116789d7b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.203185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 955/1873] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:42.204327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.215444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621025eb-53f9-4972-879e-1b830d8f473b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.220756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.221499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.221850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.222189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] PATH_INFO: `/v3/projects/e9d47b8817ff4218b556e31cb6bd8cfb/users/9cc7c1b829a649e482e61b9e08f31d54/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.238389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d47b8817ff4218b556e31cb6bd8cfb, user_id=9cc7c1b829a649e482e61b9e08f31d54, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.241129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.255480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 919/1874] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:42 2022] PUT /identity/v3/projects/e9d47b8817ff4218b556e31cb6bd8cfb/users/9cc7c1b829a649e482e61b9e08f31d54/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:42.268800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.268800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.268800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.297283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] MFA Rules not processed for user `9cc7c1b829a649e482e61b9e08f31d54`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:42.298509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:42.373614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 956/1875] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:42.378008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.486430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b13ae-f42f-4335-a6cf-916b78120bc8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.499410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.500234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.500789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.501276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.502145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.508470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.551598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 920/1876] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:42 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:42.552964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.563611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7c93a-a5c8-4f2a-bd14-ce1d67490310 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.618427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.619024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.619301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.619572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.620198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.622180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.643985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 957/1877] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:42 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:42.648853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.658931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6c220-7852-45db-b024-df002469c6bd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.663540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.664162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.664444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.664741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.665408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.667010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.677246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 921/1878] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:42.681254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.696577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbac5a-d5e4-4a3d-88cc-aa11d30f49bf admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.704691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.705432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.705718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.706136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] PATH_INFO: `/v3/projects/c16852410add409e9bc9c916449f5949/users/84581c0fa1e64c88b31d7168af6dfe90/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.723563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16852410add409e9bc9c916449f5949, user_id=84581c0fa1e64c88b31d7168af6dfe90, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.726379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.739637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 958/1879] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:42 2022] PUT /identity/v3/projects/c16852410add409e9bc9c916449f5949/users/84581c0fa1e64c88b31d7168af6dfe90/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:42.743474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.748522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:42.754755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9dabd-a320-4304-98ba-fd19466244f7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.812843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.813452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.813735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.814010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.814652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.816282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ae3b8-a411-49c9-b66f-1ce232c75b60 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.826863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 922/1880] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:42.831016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:42.837523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:42.841432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad295160-2f0c-4d14-8aac-b4fd926e91f9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:42.846949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:42.847592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.847900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.848174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] PATH_INFO: `/v3/projects/c16852410add409e9bc9c916449f5949/users/84581c0fa1e64c88b31d7168af6dfe90/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.850101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16852410add409e9bc9c916449f5949, user_id=84581c0fa1e64c88b31d7168af6dfe90, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:42.852752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:42.862780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 959/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:42 2022] PUT /identity/v3/projects/c16852410add409e9bc9c916449f5949/users/84581c0fa1e64c88b31d7168af6dfe90/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:42.873382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:42.873725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:42.874002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:42.906245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] MFA Rules not processed for user `84581c0fa1e64c88b31d7168af6dfe90`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:42.907663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:42.997551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 923/1882] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:43.010005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:43.011799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:43.078051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a166-220f-4be2-9574-b37177e2f8e5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:43.085096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:43.086672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:43.087193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:43.087683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:43.098595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:43.100090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:43.112646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 960/1883] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:43.171690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:43.181260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc39e1-0936-4a8f-a0d1-6751eb70608e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:43.185140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:43.185821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:43.186153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:43.186532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:43.292855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:43.294614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff176-cba9-4677-90b8-4079b19062ee service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:43.303810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 924/1884] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:47.960830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:47.962240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:48.031106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b0585-b8b3-41ec-908e-4a8bd16528ae service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:48.037024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:48.037610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:48.037886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:48.038150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] PATH_INFO: `/v3/users/e75494b8fc134fb19149387c947aa9e1` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:48.039851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75494b8fc134fb19149387c947aa9e1)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:48.041815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:48.126160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75494b8fc134fb19149387c947aa9e1'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:48.171327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75494b8fc134fb19149387c947aa9e1'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:48.181364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75494b8fc134fb19149387c947aa9e1'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:48.193545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75494b8fc134fb19149387c947aa9e1'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:48.196260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75494b8fc134fb19149387c947aa9e1'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:48.219566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 961/1885] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:47 2022] DELETE /identity/v3/users/e75494b8fc134fb19149387c947aa9e1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:48.221968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:48.221968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:48.222541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:48.258817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:48.261023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:48.283796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:48.367284 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 925/1886] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:48.376634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:48.378471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:48.403479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:48.498641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba328396-9d98-4c39-ab8a-e25c9855f6fa admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:48.504281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:48.505009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:48.505301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:48.505965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:48.521345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:48.524215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:48.545171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 962/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:48.972819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:48.974436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:48.987253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080c4f1-a867-4468-bc1d-7c8bdfe2b7bf None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:48.992830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:48.993637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:48.993957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:48.994226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] PATH_INFO: `/v3/projects/b81305bdc7934727b5c8e03390dc1f12` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:48.995504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81305bdc7934727b5c8e03390dc1f12)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:48.998425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:49.045739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81305bdc7934727b5c8e03390dc1f12', 'request_id': 'req-32ec7af5-728a-4c63-8bdf-1c991c3b813f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.051168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81305bdc7934727b5c8e03390dc1f12', 'request_id': 'req-32ec7af5-728a-4c63-8bdf-1c991c3b813f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.058595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81305bdc7934727b5c8e03390dc1f12', 'request_id': 'req-32ec7af5-728a-4c63-8bdf-1c991c3b813f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.069551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] The token cache is being invalidate because project b81305bdc7934727b5c8e03390dc1f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:49.069836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.070473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.071956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 926/1888] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:48 2022] DELETE /identity/v3/projects/b81305bdc7934727b5c8e03390dc1f12 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:49.075926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:49.085951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539916d-f32a-47f7-9a8a-1d614df5aa23 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:49.150909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:49.151544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.151820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.152087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] PATH_INFO: `/v3/users/a6ba8260f1684411a962d842eb5c9444` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.153225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ba8260f1684411a962d842eb5c9444)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:49.155121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:49.207621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ba8260f1684411a962d842eb5c9444'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.214764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ba8260f1684411a962d842eb5c9444'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.227725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ba8260f1684411a962d842eb5c9444'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.234301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ba8260f1684411a962d842eb5c9444'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.234928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0ae16456-ff8a-40c9-b60b-997977b15fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ba8260f1684411a962d842eb5c9444'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.240788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 963/1889] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:49 2022] DELETE /identity/v3/users/a6ba8260f1684411a962d842eb5c9444 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:49.516441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:49.518422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:49.535485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec7af5-728a-4c63-8bdf-1c991c3b813f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:49.595133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:49.595690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.596046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.596322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] PATH_INFO: `/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.597511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4b0daaaee64f44b4a9562cff7eee14)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:49.600103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:49.658673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:49.660086 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4b0daaaee64f44b4a9562cff7eee14', 'request_id': 'req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.667796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4b0daaaee64f44b4a9562cff7eee14', 'request_id': 'req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.679252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4b0daaaee64f44b4a9562cff7eee14', 'request_id': 'req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.691394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] The token cache is being invalidate because project 0d4b0daaaee64f44b4a9562cff7eee14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:49.691624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.692245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-53874fc5-b29c-4df1-9de9-f32f52a8f3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:49.693845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 927/1890] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:49 2022] DELETE /identity/v3/projects/0d4b0daaaee64f44b4a9562cff7eee14 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:49.699157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.699468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.699741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.720166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:49.721411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:49.753578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:49.794148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 964/1891] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:49.796150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e94458-3948-40e4-b68a-57556745197a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.796497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e94458-3948-40e4-b68a-57556745197a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.796826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e94458-3948-40e4-b68a-57556745197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.817358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-22e94458-3948-40e4-b68a-57556745197a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:49.818557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-22e94458-3948-40e4-b68a-57556745197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:49.826359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 928/1892] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:49.830734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:49.840822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eb143-23c8-4c4b-91bd-3454068c9942 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:49.845398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:49.845932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.846208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.846477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.847113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:49.847410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:49.849047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:49.862001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 965/1893] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:52:49.866059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e94458-3948-40e4-b68a-57556745197a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:49.875492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e94458-3948-40e4-b68a-57556745197a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:49.880151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:49.880780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.881066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.881335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.882122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:49.884019 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:49.921901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 929/1894] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:49 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:49.926088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:49.938132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3714-19cc-45f1-ab92-7d90d8effe89 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:49.996355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:49.996997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:49.997275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:49.997567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:49.998200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.000097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.026184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 966/1895] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:49 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:50.027332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.038411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e56a6-c9fd-45bd-a6f9-15539fc88de1 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.043013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.043671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.044090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.044225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.044870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.046715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.056834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 930/1896] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:50.060939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.071170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f30fc-fd6e-478f-a821-af9feb1d9b2f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.075700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.076285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.076869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.076869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.077466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.079036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca14b42-e2a4-4fbe-b2a6-b9a9ebbaea30 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.089900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 967/1897] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:50.093958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.103973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247eb3-9578-4d4d-ad9a-83958c741d57 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.108557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.109235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.109521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.109804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] PATH_INFO: `/v3/projects/b80841ed47184d1d8fffeda12075edde/users/2ae525a22e4a402f91b1508456bf33e2/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.124021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80841ed47184d1d8fffeda12075edde, user_id=2ae525a22e4a402f91b1508456bf33e2, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.126783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.140298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 931/1898] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:50 2022] PUT /identity/v3/projects/b80841ed47184d1d8fffeda12075edde/users/2ae525a22e4a402f91b1508456bf33e2/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:50.151360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.151641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.151909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.178816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] MFA Rules not processed for user `2ae525a22e4a402f91b1508456bf33e2`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:50.180014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:50.246780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 968/1899] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:50.251667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.320843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9627-6b24-4765-92db-6040a9a25eb7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.325568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.326202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.326485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.326761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.327391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.329365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.355156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 932/1900] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:50 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:50.359137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.370196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae24945-8ed1-49b1-a48c-7780b2c60a7b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.400147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:50.427889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.428496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.429297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.429584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.430208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.432122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.452965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 969/1901] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:50 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:50.456984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.467133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f08-3b05-48e9-a1fd-869a7ac06711 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.471645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.472238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.472514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.472819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.473450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.475092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.485303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 933/1902] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:50.489394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.499569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01456e1f-a9ee-49a7-9826-70ac292282f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.504067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.504726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.505003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.505272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] PATH_INFO: `/v3/projects/314a632bb6624e118bbc7bcc5db05e1a/users/f2049fe10192484db1fdcdde6b0e229e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.519877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314a632bb6624e118bbc7bcc5db05e1a, user_id=f2049fe10192484db1fdcdde6b0e229e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.522799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.535271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 970/1903] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:50 2022] PUT /identity/v3/projects/314a632bb6624e118bbc7bcc5db05e1a/users/f2049fe10192484db1fdcdde6b0e229e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:50.539145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.548853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e51e5-3b7b-44b1-82ae-d0477975ba3e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.591754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:50.609733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.610265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.610544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.610810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.611445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.613078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1ee1d-a8eb-4ba6-9989-ec8aac00482b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.622906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 934/1904] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:50.627852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.641856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14488a65-fb55-44c1-93a3-7818380125a5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.647082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.648447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.648751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.649022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] PATH_INFO: `/v3/projects/314a632bb6624e118bbc7bcc5db05e1a/users/f2049fe10192484db1fdcdde6b0e229e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.651107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] RBAC: Authorizing `identity:create_grant(project_id=314a632bb6624e118bbc7bcc5db05e1a, user_id=f2049fe10192484db1fdcdde6b0e229e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.653982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.666126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 971/1905] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:50 2022] PUT /identity/v3/projects/314a632bb6624e118bbc7bcc5db05e1a/users/f2049fe10192484db1fdcdde6b0e229e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:50.677370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.677654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.677925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.705980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] MFA Rules not processed for user `f2049fe10192484db1fdcdde6b0e229e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:50.707190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:50.775302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 935/1906] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:50.786404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.788122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:50.844582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abeb290-e2f7-4fc0-8bd6-d9383641513a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.848113 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.848746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.849022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.849288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:50.858513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:50.859980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:50.869593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 972/1907] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:50.948107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:50.960539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0ab64-657c-42c1-b9e2-93f48d517ba4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:50.964314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:50.965185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:50.965571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:50.966106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:51.031157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:51.032725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0307e5-ffdb-4f50-9db9-fbf70a76ce89 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:51.042616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 936/1908] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:53.194070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:53.195464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:53.259231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c44ba2-f8dc-48f6-9c73-b66eb477a573 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:53.263685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:53.264305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:53.264578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:53.264878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] PATH_INFO: `/v3/users/c5085765482148a79ddf0cca3c85bee5` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:53.265978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5085765482148a79ddf0cca3c85bee5)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:53.267874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:53.296973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:53.317800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5085765482148a79ddf0cca3c85bee5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.327932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5085765482148a79ddf0cca3c85bee5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.340740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5085765482148a79ddf0cca3c85bee5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.350047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5085765482148a79ddf0cca3c85bee5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.350748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5085765482148a79ddf0cca3c85bee5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.357236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 973/1909] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:53 2022] DELETE /identity/v3/users/c5085765482148a79ddf0cca3c85bee5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:53.362038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:53.362524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:53.362611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:53.384244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:53.385484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:53.450272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 937/1910] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:53.459949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:53.461380 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:53.519684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75700424-dbcc-412b-a7d1-5577bd03a80a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:53.523563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:53.524169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:53.524449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:53.524759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:53.534604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:53.536182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:53.546078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 974/1911] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:53.832193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:53.834492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:53.861762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33938-da6a-460e-838f-c0875f71e515 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:53.882895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:53.883732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:53.884264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:53.884634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] PATH_INFO: `/v3/projects/297ad9e859224973ae21f156ddc39652` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:53.887916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=297ad9e859224973ae21f156ddc39652)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:53.890561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:53.894851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:53.990698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297ad9e859224973ae21f156ddc39652', 'request_id': 'req-09f61008-3580-4559-95a5-253dff2501a9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:53.997060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297ad9e859224973ae21f156ddc39652', 'request_id': 'req-09f61008-3580-4559-95a5-253dff2501a9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.005177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297ad9e859224973ae21f156ddc39652', 'request_id': 'req-09f61008-3580-4559-95a5-253dff2501a9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.020827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] The token cache is being invalidate because project 297ad9e859224973ae21f156ddc39652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:54.021341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.022101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.024321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 938/1912] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:53 2022] DELETE /identity/v3/projects/297ad9e859224973ae21f156ddc39652 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:54.028831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:54.040088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10bcf6-c629-426d-824b-cf30a73bfcac service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:54.115924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:54.117050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:54.117690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:54.117916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] PATH_INFO: `/v3/users/9cc7c1b829a649e482e61b9e08f31d54` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:54.119916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc7c1b829a649e482e61b9e08f31d54)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:54.124630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:54.189417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc7c1b829a649e482e61b9e08f31d54'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.193257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:54.202342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc7c1b829a649e482e61b9e08f31d54'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.220333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc7c1b829a649e482e61b9e08f31d54'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.229618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc7c1b829a649e482e61b9e08f31d54'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.230441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc7c1b829a649e482e61b9e08f31d54'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.237180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 975/1913] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:54 2022] DELETE /identity/v3/users/9cc7c1b829a649e482e61b9e08f31d54 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:54.661903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:54.667173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:54.680115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f61008-3580-4559-95a5-253dff2501a9 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:54.749562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:54.750209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:54.750493 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:54.750779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] PATH_INFO: `/v3/projects/e9d47b8817ff4218b556e31cb6bd8cfb` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:54.752042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d47b8817ff4218b556e31cb6bd8cfb)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:54.754109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:54.822144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d47b8817ff4218b556e31cb6bd8cfb', 'request_id': 'req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.830832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d47b8817ff4218b556e31cb6bd8cfb', 'request_id': 'req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.838574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d47b8817ff4218b556e31cb6bd8cfb', 'request_id': 'req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.849607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] The token cache is being invalidate because project e9d47b8817ff4218b556e31cb6bd8cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:54.849893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.850553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:54.852041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 939/1914] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:54 2022] DELETE /identity/v3/projects/e9d47b8817ff4218b556e31cb6bd8cfb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:54.856139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:54.857580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:54.870501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f557d3f-65c8-48ca-aaa1-932d8210a16e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:54.899971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:54.901428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:54.925204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:54.939764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:54.940638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:54.941124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:54.941465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] PATH_INFO: `/v3/users/84581c0fa1e64c88b31d7168af6dfe90` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:54.942651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84581c0fa1e64c88b31d7168af6dfe90)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:54.945409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:54.984437 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bccf8-2f9e-42f6-a0e7-c8a32a12eaeb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:54.994302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:54.994918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:54.995198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:54.995471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] PATH_INFO: `/v3/users/c6d61b270b704bb68045f11e7f5cb48d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:54.998858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d61b270b704bb68045f11e7f5cb48d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.001387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.010232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84581c0fa1e64c88b31d7168af6dfe90'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.028992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84581c0fa1e64c88b31d7168af6dfe90'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.045046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84581c0fa1e64c88b31d7168af6dfe90'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.056330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84581c0fa1e64c88b31d7168af6dfe90'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.057743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47e8bee5-a7dd-442b-8da6-b44dd000c010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84581c0fa1e64c88b31d7168af6dfe90'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.065466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 976/1915] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:54 2022] DELETE /identity/v3/users/84581c0fa1e64c88b31d7168af6dfe90 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:55.070506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d61b270b704bb68045f11e7f5cb48d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.078686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d61b270b704bb68045f11e7f5cb48d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.093657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d61b270b704bb68045f11e7f5cb48d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.103495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d61b270b704bb68045f11e7f5cb48d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.104416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d61b270b704bb68045f11e7f5cb48d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.112291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 940/1916] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:54 2022] DELETE /identity/v3/users/c6d61b270b704bb68045f11e7f5cb48d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:55.117048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.117913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.117913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.151482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:55.154691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.213690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:55.243239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 977/1917] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:55.249321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.251046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.328047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeed68-1eb4-4f2e-8eca-fa8fa86f72ad admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.331722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.332409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.332787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.333061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.342040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.343485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafd229-cf9d-421c-a6ce-61b4d9234b66 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.353594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 941/1918] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:55.399357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.401645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.418822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e40f9a-a965-4a80-bf7b-40fc4c24c5f5 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.424053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.424987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.425268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.425535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] PATH_INFO: `/v3/projects/c16852410add409e9bc9c916449f5949` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.426924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16852410add409e9bc9c916449f5949)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.429656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.481862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16852410add409e9bc9c916449f5949', 'request_id': 'req-b8147d6c-b3fa-4641-a3ab-9ee796965be4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.487502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16852410add409e9bc9c916449f5949', 'request_id': 'req-b8147d6c-b3fa-4641-a3ab-9ee796965be4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.494545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16852410add409e9bc9c916449f5949', 'request_id': 'req-b8147d6c-b3fa-4641-a3ab-9ee796965be4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.508445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] The token cache is being invalidate because project c16852410add409e9bc9c916449f5949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:55.508779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.509423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b8147d6c-b3fa-4641-a3ab-9ee796965be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.511366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 978/1919] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:55 2022] DELETE /identity/v3/projects/c16852410add409e9bc9c916449f5949 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:55.519135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.520524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.521233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.568425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:55.570284 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.612307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:55.639156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 942/1920] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:55.650894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.651376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.651898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.679330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.680083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:55.681462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.681731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.692950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 979/1921] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:55.697658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.699780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f5c696-1fbd-4397-9b43-055be0da6d81 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.706951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce38702-596d-4c3e-837f-ee15203b2253 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.707684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.713498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.714168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.714705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.715038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] PATH_INFO: `/v3/projects/d885eb63a0134bcc9259a9170f9043d5` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.715664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.716174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.716747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.717509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d885eb63a0134bcc9259a9170f9043d5)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.717830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:55.718338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.721066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.721417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.734702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 980/1922] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:52:55.738900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.749327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a1ca4-8462-4a81-ae27-2a7a77d1347b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.754121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.754786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.755132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.755472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.756216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.758932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.781947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd885eb63a0134bcc9259a9170f9043d5', 'request_id': 'req-32217f15-01d6-46ee-8e30-c0d77cbdbc21'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.789557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd885eb63a0134bcc9259a9170f9043d5', 'request_id': 'req-32217f15-01d6-46ee-8e30-c0d77cbdbc21'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.795705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 981/1923] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:55 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:55.798980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd885eb63a0134bcc9259a9170f9043d5', 'request_id': 'req-32217f15-01d6-46ee-8e30-c0d77cbdbc21'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.800876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.813449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] The token cache is being invalidate because project d885eb63a0134bcc9259a9170f9043d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:55.813738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.814404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.816044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 943/1924] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:55 2022] DELETE /identity/v3/projects/d885eb63a0134bcc9259a9170f9043d5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:55.820638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.822042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.879970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d860161-8cbe-4ad6-a1d6-393605124228 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.885250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.885925 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.886199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.886471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.887107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.889036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.889140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217f15-01d6-46ee-8e30-c0d77cbdbc21 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.893752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.894374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.894649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.894917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] PATH_INFO: `/v3/users/e1d793209e4643ae8cc42df190025096` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.896295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d793209e4643ae8cc42df190025096)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.898309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.911954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 982/1925] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:55 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:55.915769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.917361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:55.930660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1d81-7132-4183-9ea0-1b05fde2da07 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.935722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.936346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.936862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.937130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.937877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.939342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.947156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d793209e4643ae8cc42df190025096'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.953524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 983/1926] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:55.955423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d793209e4643ae8cc42df190025096'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.957438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:55.972873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:55.978534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb120f-623a-4d98-83cb-e4ad0476a543 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:55.981371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d793209e4643ae8cc42df190025096'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:55.984718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:55.985337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:55.985615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:55.985885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:55.986530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:55.988536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:55.990231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:56.001218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d793209e4643ae8cc42df190025096'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.002261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-222db67f-2e71-47ec-8e6b-adf3ccbbddc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d793209e4643ae8cc42df190025096'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.004774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 984/1927] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:56.006080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.008888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.011616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 944/1928] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:52:55 2022] DELETE /identity/v3/users/e1d793209e4643ae8cc42df190025096 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:56.022703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766b5f7-ef68-4870-bb57-09d6880eac1e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.081075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.081744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.082024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.082300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] PATH_INFO: `/v3/projects/2cecc52ac19842f0af235b8adcf3f701/users/609422118c96443baaaeee2da80ccfcc/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.097966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cecc52ac19842f0af235b8adcf3f701, user_id=609422118c96443baaaeee2da80ccfcc, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.101325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.113964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 985/1929] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:56 2022] PUT /identity/v3/projects/2cecc52ac19842f0af235b8adcf3f701/users/609422118c96443baaaeee2da80ccfcc/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:56.126030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.126316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.126592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.177853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] MFA Rules not processed for user `609422118c96443baaaeee2da80ccfcc`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:56.179676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.244785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 945/1930] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:56.249378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.303716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.305588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.326883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c062ae-bbe8-4f1e-9d10-d92906ef2bc4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.344831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.344831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.344831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.344831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.346473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.349505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.382298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 986/1931] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:56.386121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.404388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a534947-cedb-4b85-8e8b-72d689951bf4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.452367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0f416-1fb5-47bf-9347-48325f7819e0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.457484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.458307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.458507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.458975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.459690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.462312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.472371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.473293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:56.476793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.477614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.477614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] PATH_INFO: `/v3/projects/a999b170c9224be6a7c365f8cf941def` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.479214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a999b170c9224be6a7c365f8cf941def)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.482440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.493632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 987/1932] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:56.494639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.508440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bcce4-c466-4b45-bb7c-c1a8824fb0ec admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.514059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.514834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.515118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.515700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.516650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.518047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.529220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:56.547251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 988/1933] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:56.548376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.558739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a999b170c9224be6a7c365f8cf941def', 'request_id': 'req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.569885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a999b170c9224be6a7c365f8cf941def', 'request_id': 'req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.580939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a999b170c9224be6a7c365f8cf941def', 'request_id': 'req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.583913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84743ff9-fe8d-419d-93d5-79e8eaab1aac admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.594243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] The token cache is being invalidate because project a999b170c9224be6a7c365f8cf941def was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:52:56.594543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.595406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f5b1faf0-e2ed-4047-a788-c1928a99c3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:52:56.598283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 946/1934] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:52:56 2022] DELETE /identity/v3/projects/a999b170c9224be6a7c365f8cf941def => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:56.608306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c761a-b4f5-4088-9ad8-aa1c2f760d6d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.608591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c761a-b4f5-4088-9ad8-aa1c2f760d6d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.609008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c761a-b4f5-4088-9ad8-aa1c2f760d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.632702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5c5c761a-b4f5-4088-9ad8-aa1c2f760d6d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:56.633820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5c5c761a-b4f5-4088-9ad8-aa1c2f760d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.646655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.647901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.648244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.648579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.649305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.651009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.671074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 989/1935] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:56.672134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.674405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.687934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c260-df78-4458-9dfd-2e1275c5efe0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.692565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.693258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.693540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.693815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] PATH_INFO: `/v3/projects/aad9aa49bbbf40128d922a30bd864494/users/733da8ed2ec84f5793a90a68d2e141dd/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.695286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 947/1936] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:56.702808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.703158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.703525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.710592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad9aa49bbbf40128d922a30bd864494, user_id=733da8ed2ec84f5793a90a68d2e141dd, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.713290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1061827-383b-4824-9554-3001a554bd4a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.723444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:56.724892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.725748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 990/1937] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:56 2022] PUT /identity/v3/projects/aad9aa49bbbf40128d922a30bd864494/users/733da8ed2ec84f5793a90a68d2e141dd/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:56.735792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.736011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.736300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.767245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] MFA Rules not processed for user `733da8ed2ec84f5793a90a68d2e141dd`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:56.768859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:56.790785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 948/1938] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:56.795322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.804245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50736-9d6c-4229-b73b-00e360e6f832 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.808877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.809467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.809749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.810021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.810660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:52:56.810966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.812559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.824427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 949/1939] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:52:56.828592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.837829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 991/1940] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:56.841481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e80b9-8ae8-4fc5-9a3d-09009625e92b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.842239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.843034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.843616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.843895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.844167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.844833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.846719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.853822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f729f4-0aa3-46df-834c-200c07bbe8f2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.859375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.859802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.860149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.860489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.861527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.863272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.879306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 950/1941] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:56.884463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.893754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 992/1942] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:56.898996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:56.910839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:56.928691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:56.969415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12024-c2f1-464b-b27b-d2b11063c6b7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.973067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a97e0c-7f31-4143-9d98-fe4d68bc5bb8 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:56.974012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.974617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.974892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.975161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.975849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.977790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:56.978236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:56.978842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:56.979119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:56.979390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:56.980020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:56.982066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.000861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 951/1943] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:57.004811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.009444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 993/1944] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:56 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:57.010405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.015537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397dac-cf03-4bf8-bd2f-5f1a477da8bd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.021076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44903c5-c02c-4b34-9252-4582a0e68fd9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.021559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.022172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.023118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.023391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.024030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.025736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.028336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.028973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.029252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.029519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.030154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.032051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.037011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 952/1945] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:57.040995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.044306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 994/1946] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:57.048287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.052339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497ee7d-9b04-474f-8c0c-cd3d745c007f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.068773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.068773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.068773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.068773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.069153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2474c6-65da-438a-bf74-8224af6468b7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.069767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.069767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.069961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.069961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.069961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.069961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] PATH_INFO: `/v3/projects/a45a5c3abea744f6a86b8b93b76c5120/users/0665ec21b0bb440bbb8011811fd80fd6/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.073189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 953/1947] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:57.077185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.085362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45a5c3abea744f6a86b8b93b76c5120, user_id=0665ec21b0bb440bbb8011811fd80fd6, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.086760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442890e2-9cec-4dc1-9ae8-e357b1514c78 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.088020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.091850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.092585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.093021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.093358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] PATH_INFO: `/v3/projects/7aef7c54854449cca6b99b5ab47ea570/users/811b87f30f084bdfa3606ec37c25bb17/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.101885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 995/1948] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:57 2022] PUT /identity/v3/projects/a45a5c3abea744f6a86b8b93b76c5120/users/0665ec21b0bb440bbb8011811fd80fd6/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:57.105664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.110957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aef7c54854449cca6b99b5ab47ea570, user_id=811b87f30f084bdfa3606ec37c25bb17, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.113653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a0dc0-cc4d-4738-b308-6b0046014c41 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.126932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 954/1949] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:57 2022] PUT /identity/v3/projects/7aef7c54854449cca6b99b5ab47ea570/users/811b87f30f084bdfa3606ec37c25bb17/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:57.139342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.139621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.139899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.169219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] MFA Rules not processed for user `811b87f30f084bdfa3606ec37c25bb17`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:57.170476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.174469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56caf9-ee3c-4097-90f9-46441e16dc6b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.232557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.233211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.233587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 955/1950] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:57.233728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.233822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.234530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.236123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.237921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.246857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 996/1951] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:57.248708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45a144-b781-4efc-a6b4-956dc429a21c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.251062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.253385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.253975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.254257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.254528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.255150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.257186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb15bb-3ca9-4169-b5b9-91c67eca4839 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.260867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84503d4a-6028-491d-b26c-de3d7c16b345 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.265406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.266035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.266656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.266656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] PATH_INFO: `/v3/projects/a45a5c3abea744f6a86b8b93b76c5120/users/0665ec21b0bb440bbb8011811fd80fd6/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.268440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45a5c3abea744f6a86b8b93b76c5120, user_id=0665ec21b0bb440bbb8011811fd80fd6, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.271467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.282602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 997/1952] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:57 2022] PUT /identity/v3/projects/a45a5c3abea744f6a86b8b93b76c5120/users/0665ec21b0bb440bbb8011811fd80fd6/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:57.284810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 956/1953] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:52:57 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:52:57.289577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.294328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.294611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.294929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.299225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:57.322070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] MFA Rules not processed for user `0665ec21b0bb440bbb8011811fd80fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:57.323275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.339958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:57.355166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7660bae-ee1d-4802-876b-d1e7a7dd444c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.393601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 957/1954] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:52:57.406825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.408174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.409764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.410375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.410654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.410927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.411556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.414181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.438084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 998/1955] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:52:57 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:52:57.442068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.452010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c0a7-8e5b-4987-9fe0-23f5845da27f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.456586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.457457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.457457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.457730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.458355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.460479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.468866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054e6f-b1fb-4247-bfb7-e868037f120a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.473277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 999/1956] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:52:57.474170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.475281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.475938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.476252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.476585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.478423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.483852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc06929-e426-4947-8086-d21bc414fbdb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.488522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.489183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.489462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.489735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] PATH_INFO: `/v3/projects/4e8b2343216948bebbc462aa6690114e/users/c69fd76f661e449694fcd06a1db6de8f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.504261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8b2343216948bebbc462aa6690114e, user_id=c69fd76f661e449694fcd06a1db6de8f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.507397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.521814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1000/1957] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:57 2022] PUT /identity/v3/projects/4e8b2343216948bebbc462aa6690114e/users/c69fd76f661e449694fcd06a1db6de8f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:52:57.526491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.544398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.545928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.555750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 958/1958] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:52:57.602349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f21998-1b54-476f-8a94-a2d96f6f5721 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.607539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.608224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.608720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.609303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.609898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.611576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c36eb-5afe-4b29-9c1e-a8b64d53459f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.625277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1001/1959] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:52:57.626951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.642716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35997592-a56a-4e87-9a44-106018718735 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.648471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.649361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.649728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.650065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] PATH_INFO: `/v3/projects/4e8b2343216948bebbc462aa6690114e/users/c69fd76f661e449694fcd06a1db6de8f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.652717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8b2343216948bebbc462aa6690114e, user_id=c69fd76f661e449694fcd06a1db6de8f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.654854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.681253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 959/1960] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:52:57 2022] PUT /identity/v3/projects/4e8b2343216948bebbc462aa6690114e/users/c69fd76f661e449694fcd06a1db6de8f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:52:57.683084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.683705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.684049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.697551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:57.741478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] MFA Rules not processed for user `c69fd76f661e449694fcd06a1db6de8f`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:52:57.742729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.850701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1002/1961] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:52:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:52:57.862250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:57.864170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:52:57.930155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6676d-bcdf-4a9b-b209-94615ea5c6c4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:57.934355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:57.935200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:57.935605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:57.935925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:57.951156 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:57.952187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:57.957500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:52:57.965240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 960/1962] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:52:58.059521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:52:58.068756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868eac79-3ec6-4993-bacf-b4c6dddb9561 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:52:58.072325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:52:58.072955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:52:58.073232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:52:58.073607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:52:58.131632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:52:58.133200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:52:58.142297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1003/1963] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:52:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:02.391790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:02.401342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee2607-faa3-4cbb-88f3-31e5d133eb77 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:02.405073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:02.405673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:02.405952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:02.406223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:02.462874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:02.464347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538a055-7dd8-4080-87df-0c2cf36e0120 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:02.474318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 961/1964] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:02.954066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:03.021883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd954e75-6885-4953-865e-b22acb9045cc service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:03.027982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:03.029669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.030117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.030388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] PATH_INFO: `/v3/users/811b87f30f084bdfa3606ec37c25bb17` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.031901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811b87f30f084bdfa3606ec37c25bb17)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:03.033606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:03.075780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:03.091211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811b87f30f084bdfa3606ec37c25bb17'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.099081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811b87f30f084bdfa3606ec37c25bb17'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.113346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811b87f30f084bdfa3606ec37c25bb17'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.121461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811b87f30f084bdfa3606ec37c25bb17'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.122437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811b87f30f084bdfa3606ec37c25bb17'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.129023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1004/1965] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:02 2022] DELETE /identity/v3/users/811b87f30f084bdfa3606ec37c25bb17 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:03.138220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.138511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.138789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.161541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:03.162848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:03.174601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:03.176360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:03.223415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 962/1966] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:03.232921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:03.234254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:03.237154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b79a-a530-49d4-ae8b-b59335c2c304 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:03.242499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:03.243108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.243387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.243658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.245939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:03.311872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530c63e-5edc-4af3-b51a-cba3d5e7b2a3 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:03.319848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:03.320471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.320805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.321116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.330939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:03.331910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:03.332420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:03.333381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:03.343429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1005/1967] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:03.344049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 963/1968] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:03.636852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:03.638264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:03.652893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728e5fd-2418-4af0-b67b-225deb3c7a77 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:03.657627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:03.658319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.658603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.658876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] PATH_INFO: `/v3/projects/7aef7c54854449cca6b99b5ab47ea570` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.660007 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aef7c54854449cca6b99b5ab47ea570)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:03.661942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:03.707977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aef7c54854449cca6b99b5ab47ea570', 'request_id': 'req-ac12e475-71ae-451f-b473-cc82765d21c7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.710511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:03.714944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aef7c54854449cca6b99b5ab47ea570', 'request_id': 'req-ac12e475-71ae-451f-b473-cc82765d21c7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.723446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aef7c54854449cca6b99b5ab47ea570', 'request_id': 'req-ac12e475-71ae-451f-b473-cc82765d21c7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.736071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] The token cache is being invalidate because project 7aef7c54854449cca6b99b5ab47ea570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:03.736286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.736934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.741137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1006/1969] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:03 2022] DELETE /identity/v3/projects/7aef7c54854449cca6b99b5ab47ea570 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:03.742263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:03.752005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4060a-84a9-441c-ab68-1d1df684f56c service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:03.796558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:03.814222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:03.814907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:03.815190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:03.815457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] PATH_INFO: `/v3/users/c69fd76f661e449694fcd06a1db6de8f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:03.816641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69fd76f661e449694fcd06a1db6de8f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:03.818561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:03.893011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69fd76f661e449694fcd06a1db6de8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.903898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69fd76f661e449694fcd06a1db6de8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.921887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69fd76f661e449694fcd06a1db6de8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.930968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69fd76f661e449694fcd06a1db6de8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.931610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-13260e32-2eec-4561-90f7-56ffe72fd473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69fd76f661e449694fcd06a1db6de8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:03.938883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 964/1970] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:03 2022] DELETE /identity/v3/users/c69fd76f661e449694fcd06a1db6de8f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:04.257417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.258874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.274052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e475-71ae-451f-b473-cc82765d21c7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:04.340672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:04.341703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.342156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.342669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] PATH_INFO: `/v3/projects/4e8b2343216948bebbc462aa6690114e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.343939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8b2343216948bebbc462aa6690114e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:04.346301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:04.398999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8b2343216948bebbc462aa6690114e', 'request_id': 'req-8febaeb2-fe9d-4729-b170-d5846547a0f5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.404844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8b2343216948bebbc462aa6690114e', 'request_id': 'req-8febaeb2-fe9d-4729-b170-d5846547a0f5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.411418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8b2343216948bebbc462aa6690114e', 'request_id': 'req-8febaeb2-fe9d-4729-b170-d5846547a0f5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.425812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] The token cache is being invalidate because project 4e8b2343216948bebbc462aa6690114e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:04.426086 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.426708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.428803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1007/1971] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:04 2022] DELETE /identity/v3/projects/4e8b2343216948bebbc462aa6690114e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:04.433586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373850-7cc8-4438-bc96-0c6db06070d5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.433871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373850-7cc8-4438-bc96-0c6db06070d5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.434142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373850-7cc8-4438-bc96-0c6db06070d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.460369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e1373850-7cc8-4438-bc96-0c6db06070d5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:04.461684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e1373850-7cc8-4438-bc96-0c6db06070d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.481553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.483185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.602182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 965/1972] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:04.615219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.615852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.616332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.634595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febaeb2-fe9d-4729-b170-d5846547a0f5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:04.639524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:04.641348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.641348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.641348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] PATH_INFO: `/v3/users/d17f5a02b60c4b34b159b50db3373fde` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.649151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:04.650461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.655372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17f5a02b60c4b34b159b50db3373fde)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:04.658647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:04.662161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 966/1973] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:04.664113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.679225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163411e-2967-4e3f-8161-b001239a5ac2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:04.685096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:04.691373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:04.692321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.693312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.694093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.696116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:04.696663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:04.699015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:04.719326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 967/1974] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:04.723990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.728695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17f5a02b60c4b34b159b50db3373fde'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.736703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17f5a02b60c4b34b159b50db3373fde'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.740208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e10db-ba06-4fe0-959e-16c5ec7d8360 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:04.747324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:04.748107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.748744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.749141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.749864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:04.750882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17f5a02b60c4b34b159b50db3373fde'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.753520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:04.761137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17f5a02b60c4b34b159b50db3373fde'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.762082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-7985c456-a311-41ac-8e94-72eee07c464f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17f5a02b60c4b34b159b50db3373fde'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:04.770361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1008/1975] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:04 2022] DELETE /identity/v3/users/d17f5a02b60c4b34b159b50db3373fde => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:04.775136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.775545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.775966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.812873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 968/1976] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:04 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:04.813244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:04.814975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.817761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.820198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:04.849871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:04.929175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947be42-cf88-44ec-b55a-3f3b702107f3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:04.935484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:04.936414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:04.937017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:04.937540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:04.938626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:04.951685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1009/1977] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:04.954055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:04.961349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:04.962932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.006125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 969/1978] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:04 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:05.008399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.029101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0790b-94b8-4c25-87c6-299a190839b5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.041956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.042648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.043745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.044240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.047904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.050441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.073937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 970/1979] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:05.077663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.089885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620307bd-4d80-4255-9b57-6d5319e19ab7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.098114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b280f84-d630-4351-a206-ab8170980c66 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.098210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.098210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.098210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.098210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.098777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.101137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.104573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.105232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.105510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.106091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.116659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 971/1980] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:05.120773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.128961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.130469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a15db-f281-43b4-ba9e-692489268992 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.131333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5819d6-b46d-4c3e-b96f-6eebb8cba42a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.137958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.138838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.139323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.139789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] PATH_INFO: `/v3/projects/8e680104d067470ebd37d3432310bbaa/users/e894a35d0f5e4345b1a59e495f8b7f38/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.168748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1010/1981] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:05.196706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e680104d067470ebd37d3432310bbaa, user_id=e894a35d0f5e4345b1a59e495f8b7f38, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.200003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.225680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 972/1982] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:05 2022] PUT /identity/v3/projects/8e680104d067470ebd37d3432310bbaa/users/e894a35d0f5e4345b1a59e495f8b7f38/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:05.240138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.240424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.240794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.290698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] MFA Rules not processed for user `e894a35d0f5e4345b1a59e495f8b7f38`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:05.309741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.373364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:05.393522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.396630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.475358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1011/1983] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:05.478534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.590352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670007f-4254-489d-8868-bad98e17660d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.617243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.618401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.618689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.619641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] PATH_INFO: `/v3/users/2ae525a22e4a402f91b1508456bf33e2` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.621372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae525a22e4a402f91b1508456bf33e2)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.623920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.634824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c917a-8eda-48f3-92e4-287089b6e578 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.643657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.644393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.644782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.645467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.646078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.661450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.702296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae525a22e4a402f91b1508456bf33e2'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:05.708647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1012/1984] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:05 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:05.712581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.714104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae525a22e4a402f91b1508456bf33e2'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:05.720887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:05.739425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae525a22e4a402f91b1508456bf33e2'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:05.749324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae525a22e4a402f91b1508456bf33e2'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:05.750173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a9b43290-5707-4717-ac29-53e935284cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae525a22e4a402f91b1508456bf33e2'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:05.760910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 973/1985] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:05 2022] DELETE /identity/v3/users/2ae525a22e4a402f91b1508456bf33e2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:05.764092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.765870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.766214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.801657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:05.806301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.811731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874f0e8-9985-4f3d-a78e-de8d7c3d27c6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.885733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:05.901368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.902068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.902269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.902543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.903191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.905180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:05.915451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 974/1986] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:05.919830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.922107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.935685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7a45d-1d93-4767-b590-58ae33831bb9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:05.936555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1013/1987] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:05 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:05.937235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:05.938848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:05.943373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:05.944044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:05.944339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:05.944623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] PATH_INFO: `/v3/projects/d3938f97b00745008abd7cd4217666ec` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:05.945735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3938f97b00745008abd7cd4217666ec)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:05.947652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.000832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3938f97b00745008abd7cd4217666ec', 'request_id': 'req-2a76697b-baa2-431d-85f9-9b2288478af4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.004342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22379fb2-4b92-46e4-8815-e17744468bbe admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.007452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3938f97b00745008abd7cd4217666ec', 'request_id': 'req-2a76697b-baa2-431d-85f9-9b2288478af4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.015962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3938f97b00745008abd7cd4217666ec', 'request_id': 'req-2a76697b-baa2-431d-85f9-9b2288478af4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.028158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] The token cache is being invalidate because project d3938f97b00745008abd7cd4217666ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:06.028561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.029239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.031576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 975/1988] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:05 2022] DELETE /identity/v3/projects/d3938f97b00745008abd7cd4217666ec => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:06.032271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.033591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.059702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.059702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.059993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.060077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.061815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.111859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76697b-baa2-431d-85f9-9b2288478af4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.144216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.144878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.145414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.145740 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.146336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.148209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.148811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.152678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.162257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:06.169027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1014/1989] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:06.170473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.172835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.184126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 976/1990] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:06.185187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.198764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df737b19-4a29-4e39-ae77-34e00620324b service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.204232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.205046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.205503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.205871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] PATH_INFO: `/v3/users/e9803a3b2c8549e1a04bb6ac0ef7b4af` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.207134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9803a3b2c8549e1a04bb6ac0ef7b4af)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.209397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.211144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee5173-f397-4e2f-b436-ae66b5529675 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.223633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.225112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.225401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.225679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] PATH_INFO: `/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b/users/1c78c4819c964737b616961c0316d310/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.244726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2d3b33c4f449ddb1a517f5ddc3b75b, user_id=1c78c4819c964737b616961c0316d310, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.248295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.268860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 977/1991] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:06 2022] PUT /identity/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b/users/1c78c4819c964737b616961c0316d310/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:06.269904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.343193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:06.347729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9803a3b2c8549e1a04bb6ac0ef7b4af'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.359864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9803a3b2c8549e1a04bb6ac0ef7b4af'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.387181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9803a3b2c8549e1a04bb6ac0ef7b4af'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.387314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d772-59fd-4024-b41f-cf5c3e0c9d48 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.392546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.393199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.393551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.393788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.395277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.396979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.398265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9803a3b2c8549e1a04bb6ac0ef7b4af'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.399242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9803a3b2c8549e1a04bb6ac0ef7b4af'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.409336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 978/1992] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:06.421089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.423030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.425382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1015/1993] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:06 2022] DELETE /identity/v3/users/e9803a3b2c8549e1a04bb6ac0ef7b4af => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:06.465160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbf79b-e5f5-48fa-ac1c-53a62176c991 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.610754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.611549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.611932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.612319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] PATH_INFO: `/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b/users/1c78c4819c964737b616961c0316d310/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.614548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2d3b33c4f449ddb1a517f5ddc3b75b, user_id=1c78c4819c964737b616961c0316d310, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.617494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89dd67-c3fc-477f-a3f7-220f26c8bce3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.632543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 979/1994] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:06 2022] PUT /identity/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b/users/1c78c4819c964737b616961c0316d310/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:06.635316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.637110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.644578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.644912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.645444 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.662240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52882f7-835e-49ef-95a1-2f2a6222079f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:06.688658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] MFA Rules not processed for user `1c78c4819c964737b616961c0316d310`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:06.692202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.802218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:06.829679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 980/1995] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:06.838964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:06.842146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:06.842769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:06.843044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:06.843315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] PATH_INFO: `/v3/projects/b80841ed47184d1d8fffeda12075edde` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:06.843861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:06.845605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80841ed47184d1d8fffeda12075edde)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:06.848704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:06.933284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80841ed47184d1d8fffeda12075edde', 'request_id': 'req-73503d86-97e8-4103-9e0f-9c9a0d896722'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.942498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80841ed47184d1d8fffeda12075edde', 'request_id': 'req-73503d86-97e8-4103-9e0f-9c9a0d896722'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.977787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80841ed47184d1d8fffeda12075edde', 'request_id': 'req-73503d86-97e8-4103-9e0f-9c9a0d896722'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:06.978674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f67507-e31c-46c1-8292-1afdfb474da5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.001224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] The token cache is being invalidate because project b80841ed47184d1d8fffeda12075edde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:07.001824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.002938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.005621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1016/1996] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:06 2022] DELETE /identity/v3/projects/b80841ed47184d1d8fffeda12075edde => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:07.010135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.012034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.045182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.046006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.046834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.047216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.049616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.116387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:07.143421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73503d86-97e8-4103-9e0f-9c9a0d896722 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.166428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.167869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.176133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.176955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.177329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.179182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 981/1997] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:07.179320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] PATH_INFO: `/v3/users/f2049fe10192484db1fdcdde6b0e229e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.179878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.181353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.181692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2049fe10192484db1fdcdde6b0e229e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.185149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.197389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1553e74-d040-4f47-a682-607f4e1fdc6d service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.202644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.203293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.203575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.203850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] PATH_INFO: `/v3/projects/0ef24df1d3e7449f870a99355d7965c8` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.205291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef24df1d3e7449f870a99355d7965c8)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.207413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.272998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef24df1d3e7449f870a99355d7965c8', 'request_id': 'req-158d3a15-029f-47da-b0aa-a54f6e84db16'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.275986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:07.282107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2049fe10192484db1fdcdde6b0e229e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.285750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef24df1d3e7449f870a99355d7965c8', 'request_id': 'req-158d3a15-029f-47da-b0aa-a54f6e84db16'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.292474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2049fe10192484db1fdcdde6b0e229e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.299926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef24df1d3e7449f870a99355d7965c8', 'request_id': 'req-158d3a15-029f-47da-b0aa-a54f6e84db16'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.306029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2049fe10192484db1fdcdde6b0e229e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.314489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] The token cache is being invalidate because project 0ef24df1d3e7449f870a99355d7965c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:07.314957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.315689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.316076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2049fe10192484db1fdcdde6b0e229e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.316743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0c8dfed0-2df2-422a-828a-3cd2e7e56603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2049fe10192484db1fdcdde6b0e229e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.317881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 982/1998] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:07 2022] DELETE /identity/v3/projects/0ef24df1d3e7449f870a99355d7965c8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:07.318464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.320411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.328539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1017/1999] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:07 2022] DELETE /identity/v3/users/f2049fe10192484db1fdcdde6b0e229e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:07.329954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e182fd-5662-4a51-b2f8-d5e2f86478c4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.330245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e182fd-5662-4a51-b2f8-d5e2f86478c4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.330519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e182fd-5662-4a51-b2f8-d5e2f86478c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.353650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c2e182fd-5662-4a51-b2f8-d5e2f86478c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:07.354785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c2e182fd-5662-4a51-b2f8-d5e2f86478c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.404968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d3a15-029f-47da-b0aa-a54f6e84db16 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.461670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1018/2000] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:07.465673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.465958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.466303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.502174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.502893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:07.503220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.503600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.503972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] PATH_INFO: `/v3/users/e894a35d0f5e4345b1a59e495f8b7f38` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.504146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.506675 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e894a35d0f5e4345b1a59e495f8b7f38)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.509813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.517888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1019/2001] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:07.519426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.547654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d74b2-17f2-414c-a7e9-8ca0d7d1a655 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.554394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.555109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.555486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.555853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.556592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:07.557130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.559534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.578180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1020/2002] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:53:07.580043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.589168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e894a35d0f5e4345b1a59e495f8b7f38'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.591625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4047e7-e9a9-42b0-b820-a67ef7a3853e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.596670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.597398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.597706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.597877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e894a35d0f5e4345b1a59e495f8b7f38'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.598114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.598819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.601356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.624592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e894a35d0f5e4345b1a59e495f8b7f38'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.633674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e894a35d0f5e4345b1a59e495f8b7f38'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.634365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1021/2003] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:07 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:07.634850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-068c879b-ff55-4466-8149-8d90efee9c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e894a35d0f5e4345b1a59e495f8b7f38'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:07.638626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.640877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.645804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 983/2004] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:07 2022] DELETE /identity/v3/users/e894a35d0f5e4345b1a59e495f8b7f38 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:07.647817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.648103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.648375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.669702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:07.684799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:07.686697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.691302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:07.736991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c521b9-1a7e-413d-920a-b0a0e1463817 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:07.811376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 984/2005] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:07.823547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.829048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:07.902037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:07.902950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:07.904555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:07.905040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:07.907105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:07.909301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:07.975489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1022/2006] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:07 2022] POST /identity/v3/users => generated 292 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:07.977126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:07.982266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.000170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0a45-047c-4ed7-9fa8-5b0d6e5140d5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.009617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dddab-af59-4c7d-8087-812f2c9ca2ed admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.011760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.012443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.012758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.013034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.015821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.016628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.016961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.017205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] PATH_INFO: `/v3/projects/314a632bb6624e118bbc7bcc5db05e1a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.019492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314a632bb6624e118bbc7bcc5db05e1a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.022633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.053886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.058415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.082901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 985/2007] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:08.088554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.100651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae5001-dba2-41ff-9039-c38bddeac1c1 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.108275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314a632bb6624e118bbc7bcc5db05e1a', 'request_id': 'req-ffa940cc-508e-41ca-aaee-616ab9748f45'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.119556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314a632bb6624e118bbc7bcc5db05e1a', 'request_id': 'req-ffa940cc-508e-41ca-aaee-616ab9748f45'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.140201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314a632bb6624e118bbc7bcc5db05e1a', 'request_id': 'req-ffa940cc-508e-41ca-aaee-616ab9748f45'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.165949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] The token cache is being invalidate because project 314a632bb6624e118bbc7bcc5db05e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:08.166362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.167773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.173691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1023/2008] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:07 2022] DELETE /identity/v3/projects/314a632bb6624e118bbc7bcc5db05e1a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:08.173691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.176555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.192474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.195277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.195614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.195928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.199458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.199458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fab87-06c1-4057-8ef2-2459f5fb7719 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.214356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 986/2009] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:08.215902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.216248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.216576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.243851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:08.246256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.268777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 987/2010] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:08.270692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.275435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.299216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ebc3d-6256-48bc-b943-aaa780ac67d4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.302674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:08.311446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.312317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.312738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.313108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.319382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.323668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9628e7d-f1b5-4050-b9ea-e8cc5b3b2cc9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.324256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa940cc-508e-41ca-aaee-616ab9748f45 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.331319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.332258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.332713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.333099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] PATH_INFO: `/v3/users/609422118c96443baaaeee2da80ccfcc` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.334713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] RBAC: Authorizing `identity:delete_user(user_id=609422118c96443baaaeee2da80ccfcc)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.336946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.340144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 988/2011] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:08.342538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.343658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.344483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.403002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:08.411430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.417661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609422118c96443baaaeee2da80ccfcc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.436121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 989/2012] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:08.436121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.452218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3f238-93b1-4b5b-bd6a-fbbe2221eb37 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.452730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609422118c96443baaaeee2da80ccfcc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.457999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.458710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.460692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.460913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] PATH_INFO: `/v3/projects/b0e592d482f14552bce0f0fbd31e0c5a/users/de88c1fad9ab4f4ba5d52dd1d927098f/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.474633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609422118c96443baaaeee2da80ccfcc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.483597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609422118c96443baaaeee2da80ccfcc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.484259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609422118c96443baaaeee2da80ccfcc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:08.493893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e592d482f14552bce0f0fbd31e0c5a, user_id=de88c1fad9ab4f4ba5d52dd1d927098f, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.495791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1024/2013] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:08 2022] DELETE /identity/v3/users/609422118c96443baaaeee2da80ccfcc => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:08.496876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.498553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.501865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc463d48-4fb6-44e8-b705-7d68691cfeb9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.515759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:08.534311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 990/2014] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:08 2022] PUT /identity/v3/projects/b0e592d482f14552bce0f0fbd31e0c5a/users/de88c1fad9ab4f4ba5d52dd1d927098f/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:08.538334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf02e4-50f0-41d4-bdad-210d0f7544c4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.538671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf02e4-50f0-41d4-bdad-210d0f7544c4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.539500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf02e4-50f0-41d4-bdad-210d0f7544c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.569906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5bcf02e4-50f0-41d4-bdad-210d0f7544c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:08.572945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5bcf02e4-50f0-41d4-bdad-210d0f7544c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.605416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c5512-271c-44dd-b510-e7fe0fb6120e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.722970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.723640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.723921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.724192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.725240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:08.725989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.727823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.746966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 991/2015] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:08.747246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1025/2016] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:08.748160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.748561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.748893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.749175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.768583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec4b0d-4d84-45ac-a307-58b49915089e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.775382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.776579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.777026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:08.777295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:08.779070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:08.781363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:08.787013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] MFA Rules not processed for user `de88c1fad9ab4f4ba5d52dd1d927098f`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:08.789069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.800047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:08.836629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1026/2017] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:08.836629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.839769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:08.923806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 992/2018] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:08.923806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:08.936903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b8854-2270-4fb0-8d83-03001523599c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.989461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7ed5-45c9-442b-80e2-9311767392e7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:08.996260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:08.998982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:08.999476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.000010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.034327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.060849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.061800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.062085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.062799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.064809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.103331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.106582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.127026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1027/2019] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:09.128365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.130545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 993/2020] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:08 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:09.132369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.148477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.148721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.169608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c48e2a-bb62-4c83-bd23-f19b4f9e71a5 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.182966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.183696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.184049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.184395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] PATH_INFO: `/v3/projects/8e680104d067470ebd37d3432310bbaa` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.184642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945d092-d705-43c5-9ac6-9314afe7bb10 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.187780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e680104d067470ebd37d3432310bbaa)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.189969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.191726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.192441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.193118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.193395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.194839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.196913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.220151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:09.242193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 994/2021] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:09 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:09.243239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.309855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e680104d067470ebd37d3432310bbaa', 'request_id': 'req-2b196fbc-908e-4fa8-86f9-7547f66920f0'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.322566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e680104d067470ebd37d3432310bbaa', 'request_id': 'req-2b196fbc-908e-4fa8-86f9-7547f66920f0'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.333112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e680104d067470ebd37d3432310bbaa', 'request_id': 'req-2b196fbc-908e-4fa8-86f9-7547f66920f0'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.358746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] The token cache is being invalidate because project 8e680104d067470ebd37d3432310bbaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:09.359217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.360168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.362037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1028/2022] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:09 2022] DELETE /identity/v3/projects/8e680104d067470ebd37d3432310bbaa => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:09.362936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.366677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.368360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73790a12-a9fc-490a-9305-ec0833f857d7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.406041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:09.456292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.457146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.457529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.457991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.458820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.460644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.470413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196fbc-908e-4fa8-86f9-7547f66920f0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.472983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 995/2023] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:09.474102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.477933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.478193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.478877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.479159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.479434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.480212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.482213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.510125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1029/2024] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:09 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:09.511210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.512503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b76aa7-3ac2-4f01-9d1a-7772dd4d8259 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.514286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.518249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.519270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.519653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.520833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] PATH_INFO: `/v3/users/1c78c4819c964737b616961c0316d310` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.522408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c78c4819c964737b616961c0316d310)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.524714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.535771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44214590-ea39-4918-b9ed-1abb8d68ce48 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.543434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.544720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.545168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.545587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.547855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.550047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.567823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1030/2025] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:09.568889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.574917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.587556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c78c4819c964737b616961c0316d310'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.589360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eca53a-1a68-48c9-bf34-138c45667a4d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.594307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.595133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.595415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.595689 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] PATH_INFO: `/v3/users/4f824db60faf4f959158f112c528b463` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.596888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f824db60faf4f959158f112c528b463)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.599012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.605159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c78c4819c964737b616961c0316d310'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.625158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c78c4819c964737b616961c0316d310'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.641482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c78c4819c964737b616961c0316d310'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.642686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c78c4819c964737b616961c0316d310'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.650144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 996/2026] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:09 2022] DELETE /identity/v3/users/1c78c4819c964737b616961c0316d310 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:09.650980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.665475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.665891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f824db60faf4f959158f112c528b463'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.675955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f824db60faf4f959158f112c528b463'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.691808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:09.694868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f824db60faf4f959158f112c528b463'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.723241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f824db60faf4f959158f112c528b463'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.728718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f824db60faf4f959158f112c528b463'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:09.738337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1031/2027] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:09 2022] DELETE /identity/v3/users/4f824db60faf4f959158f112c528b463 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:09.739498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.742055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.776402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d3559-c9eb-4b89-af9c-ec39a8db7e66 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.869289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.870093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.870445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.870829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.871744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.873670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656fe04-f320-4879-9af4-4430e5095c1b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.883179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974887-3bcd-4e37-8bc7-cddceb7871de admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:09.887579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 997/2028] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:09.889076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:09.889258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.889601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.889875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:09.889996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.890155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:09.890673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] PATH_INFO: `/v3/projects/e5c383f0136f4c828cd03c80df2bee75/users/9b2872a9477e46a79e437da48103b5eb/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:09.910174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5c383f0136f4c828cd03c80df2bee75, user_id=9b2872a9477e46a79e437da48103b5eb, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:09.914340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:09.915397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:09.916782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.918194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:09.933415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 998/2029] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:09.934306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.935788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.942661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1032/2030] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:09 2022] PUT /identity/v3/projects/e5c383f0136f4c828cd03c80df2bee75/users/9b2872a9477e46a79e437da48103b5eb/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:09.944287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:09.951116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:09.956383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed135a56-e67d-432a-b518-5fb0669cb238 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.042565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.043255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.043574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.043850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] PATH_INFO: `/v3/projects/2cecc52ac19842f0af235b8adcf3f701` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.044915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055323-f9a9-4bcc-b77b-73ddc3ac9a41 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.047487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cecc52ac19842f0af235b8adcf3f701)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.049697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.054039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.057760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.061341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.061655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] PATH_INFO: `/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5/users/cf3c90263b9f4079ab5848b9cb307e9c/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.083833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c42f211cb9a4448b3368d8fcfe0c3b5, user_id=cf3c90263b9f4079ab5848b9cb307e9c, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.087727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.106055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1033/2031] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:09 2022] PUT /identity/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5/users/cf3c90263b9f4079ab5848b9cb307e9c/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:10.106917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.109628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.116980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cecc52ac19842f0af235b8adcf3f701', 'request_id': 'req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.118623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:10.125826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cecc52ac19842f0af235b8adcf3f701', 'request_id': 'req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.137210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cecc52ac19842f0af235b8adcf3f701', 'request_id': 'req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.160359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] The token cache is being invalidate because project 2cecc52ac19842f0af235b8adcf3f701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:10.160595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.161279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-42a04a1d-ffc6-4254-a00f-8d5d4a61fea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.166054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 999/2032] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:09 2022] DELETE /identity/v3/projects/2cecc52ac19842f0af235b8adcf3f701 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:10.166054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.166054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.166054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.195420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] MFA Rules not processed for user `9b2872a9477e46a79e437da48103b5eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:10.196761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.204135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0376c-7fa1-4c28-8ae6-9c6f06a3043d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.265069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.265678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.265956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.266224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.267155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1000/2033] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:10.268070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.268165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.269471 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.319979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:10.337290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f51428-c30c-42b5-a5b2-17919124f87a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.337593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.339083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.342025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.342661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.343445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.343445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] PATH_INFO: `/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.344328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2d3b33c4f449ddb1a517f5ddc3b75b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.346267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.356792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1034/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:10.359764 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.362137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.376044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe773551-a813-462e-8da2-89e0c21d8184 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.382060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.382667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.382968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.383209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.383899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.385492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.396238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1035/2035] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:10.397832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.401676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.404554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2d3b33c4f449ddb1a517f5ddc3b75b', 'request_id': 'req-00becc05-073a-4f4a-817d-1cb6519fd4ba'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.410754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2d3b33c4f449ddb1a517f5ddc3b75b', 'request_id': 'req-00becc05-073a-4f4a-817d-1cb6519fd4ba'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.414621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5792c52-5d28-4e75-bcd9-cf09e9795770 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.419823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2d3b33c4f449ddb1a517f5ddc3b75b', 'request_id': 'req-00becc05-073a-4f4a-817d-1cb6519fd4ba'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.429178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:10.438903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] The token cache is being invalidate because project ff2d3b33c4f449ddb1a517f5ddc3b75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:10.439512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.440540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.442893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1001/2036] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:10 2022] DELETE /identity/v3/projects/ff2d3b33c4f449ddb1a517f5ddc3b75b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:10.445327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.447189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.502978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.504489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.504895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.505324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] PATH_INFO: `/v3/users/733da8ed2ec84f5793a90a68d2e141dd` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.507413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=733da8ed2ec84f5793a90a68d2e141dd)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.510415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.532058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00becc05-073a-4f4a-817d-1cb6519fd4ba admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.538011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.538632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.538913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.539184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.539879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.542115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.568308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '733da8ed2ec84f5793a90a68d2e141dd'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.577985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1002/2037] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:10.578949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.580971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.584193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '733da8ed2ec84f5793a90a68d2e141dd'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.600874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '733da8ed2ec84f5793a90a68d2e141dd'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.614024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '733da8ed2ec84f5793a90a68d2e141dd'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.614726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d3f83450-76c3-47b3-a63b-d7b2cb78d2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '733da8ed2ec84f5793a90a68d2e141dd'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.626396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1036/2038] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:10 2022] DELETE /identity/v3/users/733da8ed2ec84f5793a90a68d2e141dd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:10.626396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.626396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.626396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.660140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:10.662702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.676537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b7c25-616a-47b1-a7de-07514818d0b8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.749157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:10.766720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.767902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.768778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.769252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] PATH_INFO: `/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5/users/cf3c90263b9f4079ab5848b9cb307e9c/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.773375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c42f211cb9a4448b3368d8fcfe0c3b5, user_id=cf3c90263b9f4079ab5848b9cb307e9c, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.776247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1037/2039] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:10.777316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.777961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.778644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.791775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f89bf-4af6-4456-a904-6af9e0943235 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.793895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1003/2040] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:10 2022] PUT /identity/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5/users/cf3c90263b9f4079ab5848b9cb307e9c/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:10.796153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:10.806866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.848032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:10.870382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.870710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.871105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.871573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.872339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.874471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff78f88-2b50-4b09-b4ce-17f7061ac27b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.896577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836816b-0106-4bb3-9924-ad5c7698b057 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:10.900053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1038/2041] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:10.900053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc6172-986f-4035-8324-a48a98eabf0a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.900053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc6172-986f-4035-8324-a48a98eabf0a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.900053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc6172-986f-4035-8324-a48a98eabf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.905466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:10.906489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.906489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.906685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] PATH_INFO: `/v3/projects/c0e05d458fb74e3b9e0bcd9a3e662a0f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.907813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e05d458fb74e3b9e0bcd9a3e662a0f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:10.909832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:10.920362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8ecc6172-986f-4035-8324-a48a98eabf0a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:10.921561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8ecc6172-986f-4035-8324-a48a98eabf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.933023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1039/2042] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:10.934290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:10.934569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:10.934845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:10.968323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e05d458fb74e3b9e0bcd9a3e662a0f', 'request_id': 'req-60b415ba-c63a-43a3-889f-3265ff95648c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.969155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] MFA Rules not processed for user `cf3c90263b9f4079ab5848b9cb307e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:10.970452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:10.980346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e05d458fb74e3b9e0bcd9a3e662a0f', 'request_id': 'req-60b415ba-c63a-43a3-889f-3265ff95648c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:10.992764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e05d458fb74e3b9e0bcd9a3e662a0f', 'request_id': 'req-60b415ba-c63a-43a3-889f-3265ff95648c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.005724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] The token cache is being invalidate because project c0e05d458fb74e3b9e0bcd9a3e662a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:11.005951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.006643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.008892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1004/2043] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:10 2022] DELETE /identity/v3/projects/c0e05d458fb74e3b9e0bcd9a3e662a0f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:11.009696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.011060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.050991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1040/2044] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:11.051907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.053794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.076900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b415ba-c63a-43a3-889f-3265ff95648c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.081557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.082163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.082450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.082724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.083369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.085107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.095102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1005/2045] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:11.095742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.097427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.110566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf93e3-9928-4053-bc2a-995633cef2d8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.115109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.116365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.116673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.116948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] PATH_INFO: `/v3/projects/aad9aa49bbbf40128d922a30bd864494` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.118056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad9aa49bbbf40128d922a30bd864494)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.120069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.122167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95eaca-d281-43f5-998e-f2468d1b7d6e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.126847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.127456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.127737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.128136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.128991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:11.129102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.131111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd31e8-9055-42f1-8cfa-33499c368695 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.136105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:11.150047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1041/2046] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:11.150047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.150047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.150047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.170877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:11.172020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.174970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:11.179585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad9aa49bbbf40128d922a30bd864494', 'request_id': 'req-d551a560-67e8-4055-8bc7-ab10137aca34'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.188571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad9aa49bbbf40128d922a30bd864494', 'request_id': 'req-d551a560-67e8-4055-8bc7-ab10137aca34'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.197741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad9aa49bbbf40128d922a30bd864494', 'request_id': 'req-d551a560-67e8-4055-8bc7-ab10137aca34'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.209690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] The token cache is being invalidate because project aad9aa49bbbf40128d922a30bd864494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:11.209988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.210757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.216136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1006/2047] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:11 2022] DELETE /identity/v3/projects/aad9aa49bbbf40128d922a30bd864494 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:11.216210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.220028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.248705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1042/2048] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:11.248705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.250663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.266389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a2da2-f72c-43be-858d-4891a2c3ab32 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.272360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.273116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.273462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.273805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] PATH_INFO: `/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1/users/b891e47a3959487580d682fc561bad98/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.292629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fc0f49349b433cb6c4e6ad5a12f3b1, user_id=b891e47a3959487580d682fc561bad98, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.295536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.302425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a560-67e8-4055-8bc7-ab10137aca34 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.308393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.309780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.310169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.310547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.312758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1043/2049] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:11 2022] PUT /identity/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1/users/b891e47a3959487580d682fc561bad98/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:11.312987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.313578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.316364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.392471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8ed4-2faa-4f52-9ac2-085bbbb10745 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.394535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.396367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.398841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.399459 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.399736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.400011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.400683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.403938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778696fd-de23-475e-86f4-c74eca715eb9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.413377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1007/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:11.414373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.415816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.433820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5184c-c0af-4248-9864-38eed62c1f87 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.445275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.446522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.446996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.447126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1044/2051] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:11.447446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] PATH_INFO: `/v3/users/0665ec21b0bb440bbb8011811fd80fd6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.448457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.448820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.449035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0665ec21b0bb440bbb8011811fd80fd6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.449142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.451477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.478856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:11.480088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.559053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0665ec21b0bb440bbb8011811fd80fd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.568409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0665ec21b0bb440bbb8011811fd80fd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.581344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:11.584773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0665ec21b0bb440bbb8011811fd80fd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.590986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:11.596122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0665ec21b0bb440bbb8011811fd80fd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.596466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1045/2052] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:11.597190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0665ec21b0bb440bbb8011811fd80fd6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:11.597322 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.606180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1008/2053] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:11 2022] DELETE /identity/v3/users/0665ec21b0bb440bbb8011811fd80fd6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:11.607210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.607789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee6ae4-e209-4e84-89d5-b49909744eea None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.608966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.685934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.686591 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.686850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.687146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.688107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.690067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.701711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1046/2054] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:11.703070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.705258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.728362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad40ec3-7068-425f-872d-2f7c1f5408a5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.734651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.735380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.735748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.736278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.736997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.739174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.765915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1009/2055] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/users => generated 305 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:11.766786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.769238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.770162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e6c1c-45e7-42c7-b131-c8f9f74ec9d5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.774021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.774629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.774910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.775341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.777458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.802313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975926e0-3b2a-4a99-b256-a4395f1a0459 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.811270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.812228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.812581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.813059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.814292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:11.814720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.817542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.833875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1010/2056] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:11.833875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.833875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:11.853471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.854171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c5be8-4166-46e9-8022-600c4f816f1d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.854961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.858979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.859734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.860013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.860295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] PATH_INFO: `/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1/users/b891e47a3959487580d682fc561bad98/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.862908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fc0f49349b433cb6c4e6ad5a12f3b1, user_id=b891e47a3959487580d682fc561bad98, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.865180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1047/2057] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:11.866088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.866632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.878242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48a0fd-6fa6-45c9-adbb-d9ece5a79426 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.879274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1011/2058] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:11 2022] PUT /identity/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1/users/b891e47a3959487580d682fc561bad98/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:11.890037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:11.954967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:11.974030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:11.974030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.974030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.974030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:11.975060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:11.975758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3a9f1-91c1-47d8-a0f6-0f9687a685c8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:11.984240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5ffa-f3ad-433d-b4bb-161bb096dd33 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:11.991432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1048/2059] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:11.991432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:11.991432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:11.991730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.010075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.010964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.012116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.013549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.015130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.017926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.026864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] MFA Rules not processed for user `b891e47a3959487580d682fc561bad98`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:12.028089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.057498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1012/2060] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/projects => generated 359 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:12.058503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.060435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.112570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:12.113407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1049/2061] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:12.118118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.140396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b190a9e-6d71-4bca-899b-d5b131453fa3 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.143579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d8d89-64e3-4d3e-bec9-53009d33a906 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.146472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.147292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.147589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.147858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.148550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.149153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.149908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.150192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.150306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.150458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] PATH_INFO: `/v3/projects/a45a5c3abea744f6a86b8b93b76c5120` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.158486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45a5c3abea744f6a86b8b93b76c5120)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.161303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.162594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1050/2062] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:12.162594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.175326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2207400-cc9a-4bab-be9a-dce509d0bc0c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.197204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.198204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.198839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.199327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.200241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.203561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.226810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45a5c3abea744f6a86b8b93b76c5120', 'request_id': 'req-ad50cc11-82f0-40e0-8c09-090fadbe0074'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:12.241092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1051/2063] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/users => generated 280 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:12.242862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.245129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.246333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45a5c3abea744f6a86b8b93b76c5120', 'request_id': 'req-ad50cc11-82f0-40e0-8c09-090fadbe0074'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:12.255953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45a5c3abea744f6a86b8b93b76c5120', 'request_id': 'req-ad50cc11-82f0-40e0-8c09-090fadbe0074'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:12.268567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] The token cache is being invalidate because project a45a5c3abea744f6a86b8b93b76c5120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:12.268901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:12.269698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:12.272869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1013/2064] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:12 2022] DELETE /identity/v3/projects/a45a5c3abea744f6a86b8b93b76c5120 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:12.273425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.274986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.316278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a56872-e5ac-41a6-b8c6-a1fbb70c21a8 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.342525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cc11-82f0-40e0-8c09-090fadbe0074 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.347408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.348070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.348353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.348654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] PATH_INFO: `/v3/projects/1d4601eb8dc141519a3d478cc5439e32/users/7cdab5749ce44e34b059d1386b9c7ba6/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.363997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4601eb8dc141519a3d478cc5439e32, user_id=7cdab5749ce44e34b059d1386b9c7ba6, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.366963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.368108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.368739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.369025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.369299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.371093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.381423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1014/2065] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:12 2022] PUT /identity/v3/projects/1d4601eb8dc141519a3d478cc5439e32/users/7cdab5749ce44e34b059d1386b9c7ba6/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:12.382508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.382953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:12.384130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.444510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.445069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:12.446247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b04f3-58e9-4801-90b6-99ce0898bcb2 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.449337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3b3ce-133e-4d16-9c71-17ad0ec7b410 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.455481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.456101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.456780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.457389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.457582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1052/2066] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:12.458165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.458712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.459664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.459945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.460632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa903-339b-4e75-9f7a-360d454fa84e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.471098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1015/2067] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:12.472329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.472644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.472913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.481660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:12.483072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.498289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1053/2068] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:12.498289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.502844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f65f-15ef-4b98-b736-fced947b35a7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.509218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] MFA Rules not processed for user `7cdab5749ce44e34b059d1386b9c7ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:12.509947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.510517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.510624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.510846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.511115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.511754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.514544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be083eb-91e3-46e8-9df5-e3aa105d0a54 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.529425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1054/2069] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:12.529425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.529425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.529425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.550288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:12.551510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.560789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1055/2070] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:12.561449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.562753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.589309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1016/2071] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:12.589309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.600078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6258-b0ab-4235-81a7-b3fb2a9375d8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.605332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.606064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.606434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.606781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] PATH_INFO: `/v3/projects/4a2a680830b340049cae493c83e8edb7/users/f3bee8e07dd74f8cbb82322f0ecf33bf/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.624987 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2a680830b340049cae493c83e8edb7, user_id=f3bee8e07dd74f8cbb82322f0ecf33bf, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.627813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31033939-7539-40c2-a8d7-00abc7766361 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.628295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.632636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.632881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.633265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.633381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.635462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.646269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1017/2072] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:12 2022] PUT /identity/v3/projects/4a2a680830b340049cae493c83e8edb7/users/f3bee8e07dd74f8cbb82322f0ecf33bf/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:12.647839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.718499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.720098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.730081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1056/2073] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:12.730739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.735995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7898e4-8d05-4efa-b5ce-632d3b1d966f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.744786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce284e8-4c45-41d5-8132-50a2ec7a384d service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.750540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.751302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.751644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.751984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.752734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.755016 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.775129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.775838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.776187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.776526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.777351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:12.777722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.779564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30c85d-0fb6-435d-b326-1b2bd992621b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.794764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1018/2074] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:12.798646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1057/2075] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/projects => generated 407 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:12.798781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.798781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.798781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.800345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.827374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:12.846818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] MFA Rules not processed for user `f3bee8e07dd74f8cbb82322f0ecf33bf`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:12.848069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:12.894017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b042-c200-4418-b0be-ef54891cb7e4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.899250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.899969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.900271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.900542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.901253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.903337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.908998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:12.937525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1058/2076] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/users => generated 310 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:12.939370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.945529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1019/2077] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:12.946704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:12.957963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a6ee-04ae-4404-b794-426454d63be6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.961998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e60584-977f-48c3-b83d-479befa9e235 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:12.967596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.968180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.968457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.968793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.969560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.971915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:12.972532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:12.972836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:12.973257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:12.973874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:12.975803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.979639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:12.999432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1020/2078] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:13.003706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.022860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b3bab-05ca-4738-ae46-04145d92df5c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.027963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.028848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.029312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.029649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] PATH_INFO: `/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d/users/89de26a87560404883e4affceb13a69d/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.036987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1059/2079] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:12 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:13.037857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.040805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.049364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0959ac2b4f45a0aafa1eb32003009d, user_id=89de26a87560404883e4affceb13a69d, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.052638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.074094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1021/2080] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:13 2022] PUT /identity/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d/users/89de26a87560404883e4affceb13a69d/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:13.074094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.119284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a890a-9c3e-4cab-a067-08af6feed3f5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.172295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917569a2-6936-4be3-95a5-86115b61f72a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.186700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.187544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.188025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.188420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.189775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.192673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.203654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.203654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.203654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.203654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.235309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1022/2081] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:13 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:13.236245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.249972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbdfce-c1f4-4c6a-a11b-f638f8079f3e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.256813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.257484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.257761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.258042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.259026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.260443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.272722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1023/2082] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:13.273777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.284867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63b1ef-5ddb-4bb3-86dd-f84acfc2281c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.290897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.291637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.291924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.292354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.292930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.294546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.306092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:13.306467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1024/2083] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:13.307764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.313073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.314963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.320308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c593c-2394-4762-a5bb-745e10aafa4a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.326617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.327271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1060/2084] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:13.327393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.327576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.327911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] PATH_INFO: `/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d/users/89de26a87560404883e4affceb13a69d/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.328534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.330214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0959ac2b4f45a0aafa1eb32003009d, user_id=89de26a87560404883e4affceb13a69d, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.344523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d117a04-72f2-4250-afa8-f4b81964de75 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.346867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.352882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.353577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.354132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.354487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.355321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.357382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d90eb-6dfd-4944-a432-d7ff969a6c2b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.360754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1025/2085] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:13 2022] PUT /identity/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d/users/89de26a87560404883e4affceb13a69d/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:13.361648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.389907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1061/2086] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:13.391341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.391696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.392037 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.433365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] MFA Rules not processed for user `89de26a87560404883e4affceb13a69d`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:13.434649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.446067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4aa3e-31eb-48c1-8675-22433abf791d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.452714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.453446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.453818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.454150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] PATH_INFO: `/v3/users/f3bee8e07dd74f8cbb82322f0ecf33bf` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.455578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bee8e07dd74f8cbb82322f0ecf33bf)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.457996 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.460986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:13.527895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bee8e07dd74f8cbb82322f0ecf33bf'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:13.537426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bee8e07dd74f8cbb82322f0ecf33bf'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:13.540477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1062/2087] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:13 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:13.541539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.552394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bee8e07dd74f8cbb82322f0ecf33bf'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:13.553445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e66e79-04cf-4c75-80e4-e9f4ea011980 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.561885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.562723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.563233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.563869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] PATH_INFO: `/v3/projects/62505cc4155a4e62b85ea34def321ffb/users/d0d8bda4641d4003b38c6661e152d9f6/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.564696 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bee8e07dd74f8cbb82322f0ecf33bf'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:13.565899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bee8e07dd74f8cbb82322f0ecf33bf'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:13.575767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1026/2088] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:13 2022] DELETE /identity/v3/users/f3bee8e07dd74f8cbb82322f0ecf33bf => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:13.577496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.579485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.582561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62505cc4155a4e62b85ea34def321ffb, user_id=d0d8bda4641d4003b38c6661e152d9f6, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.585490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4371bd9-1fe1-4234-a0df-3f6d0ae7e413 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.605610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1063/2089] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:13 2022] PUT /identity/v3/projects/62505cc4155a4e62b85ea34def321ffb/users/d0d8bda4641d4003b38c6661e152d9f6/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:13.610870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085466a6-5459-4493-aa8b-6cd13c06d7cb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.611292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085466a6-5459-4493-aa8b-6cd13c06d7cb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.611627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085466a6-5459-4493-aa8b-6cd13c06d7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.639013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-085466a6-5459-4493-aa8b-6cd13c06d7cb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:13.640693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-085466a6-5459-4493-aa8b-6cd13c06d7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.663962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390e50-7455-40fc-8a39-207c585b0fa2 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.723203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.724175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.724553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.724927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.726728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.733053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1064/2090] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:13.734555 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.734843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.735122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.769252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] MFA Rules not processed for user `d0d8bda4641d4003b38c6661e152d9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:13.770892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.787995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:13.821283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.823577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.828179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:13.848695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1027/2091] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:13.850343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.852732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.895654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1065/2092] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:13.915374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:13.918014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:13.958833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537f990-e825-4d53-b235-b684d9bcac83 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:13.963580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:13.964235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:13.964516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:13.964821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:13.977063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:13.978908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:13.996524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1028/2093] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:14.020524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fe04f-cedf-4d50-a508-d2764af1212c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.026464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.027428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.027716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.027991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.044144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.046245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.076403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:14.077574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1066/2094] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:14.091724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9b363-23e1-4f75-827e-4352d6f98829 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.097480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.099173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.100065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.100616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.103694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:14.213246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.214962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e42df-05f5-4447-b501-94937497e756 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.239857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1029/2095] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:14 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:14.413283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:14.415197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:14.441763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153b209-7c2d-45a7-bd3c-f1c9bea5204c service glance] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.458585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.459377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.459707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.459985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] PATH_INFO: `/v3/projects/4a2a680830b340049cae493c83e8edb7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.470525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2a680830b340049cae493c83e8edb7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.473233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.544302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2a680830b340049cae493c83e8edb7', 'request_id': 'req-360f8d1b-3fdb-4804-a852-354805aedd20'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:14.550993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2a680830b340049cae493c83e8edb7', 'request_id': 'req-360f8d1b-3fdb-4804-a852-354805aedd20'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:14.559562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2a680830b340049cae493c83e8edb7', 'request_id': 'req-360f8d1b-3fdb-4804-a852-354805aedd20'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:14.576912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] The token cache is being invalidate because project 4a2a680830b340049cae493c83e8edb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:14.577336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:14.577834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-360f8d1b-3fdb-4804-a852-354805aedd20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:14.579372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1067/2096] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:14 2022] DELETE /identity/v3/projects/4a2a680830b340049cae493c83e8edb7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:14.585015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.585321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.585670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.630136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:14.635007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:14.782400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1030/2097] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:14.791500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.792168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.792168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.807519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:14.809200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:14.846223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:14.847730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:14.849271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f2e6-2893-4116-91c6-07ef74e9901c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.869966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1068/2098] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:14.870913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:14.871852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.872878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.873325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.873746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] PATH_INFO: `/v3/users/de88c1fad9ab4f4ba5d52dd1d927098f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.875231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de88c1fad9ab4f4ba5d52dd1d927098f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.881031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.883407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf42a66-d8d2-493f-9e62-add1a97adf0e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.891906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.892819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.893102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.893498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.894198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:14.896782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.898821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.923315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1069/2099] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:14.930495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:14.940537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:14.958794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345d8e4-f164-439c-896d-013728738bcf admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:14.973242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:14.973242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:14.973242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:14.973242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:14.973620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:14.978022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:14.995678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de88c1fad9ab4f4ba5d52dd1d927098f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:15.019557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de88c1fad9ab4f4ba5d52dd1d927098f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:15.031778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1070/2100] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:14 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:15.035183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.065218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de88c1fad9ab4f4ba5d52dd1d927098f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:15.077320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de88c1fad9ab4f4ba5d52dd1d927098f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:15.079719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:15.079807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-422ae181-809d-4878-9145-15710ec9db03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de88c1fad9ab4f4ba5d52dd1d927098f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:15.093186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1031/2101] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:14 2022] DELETE /identity/v3/users/de88c1fad9ab4f4ba5d52dd1d927098f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:15.098296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.098584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.098883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.152661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:15.156832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:15.164794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa954408-bb7e-46e0-9be2-d17c12e4d977 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.275339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.276112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.277008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.277501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.279066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.282219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.304229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1032/2102] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:15.315237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1071/2103] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:15 2022] POST /identity/v3/users => generated 287 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:15.316529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.318071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:15.325661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.328144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:15.338858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6aeedc-bb8b-44de-8c35-f37b4aaa5025 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.348354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.350239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.350889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.351305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.352487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.354337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.366835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1033/2104] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:15.372071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.418477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911fbd-9cfe-4000-a18c-24b889635f32 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.425064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.425681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.425959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.426845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.427476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.429270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.445082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1034/2105] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:15.446169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd8d0-e46f-4f51-a25a-c19d25c4a340 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.446729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.453496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.455725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.456383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.456698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.459124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad4c0-1598-4a96-bd0b-f94abe3c15ce admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.464208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.465135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.465539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.465912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] PATH_INFO: `/v3/projects/9767ed3b35f84c65a1e8d8d1355154dd/users/224365c7065347d3820a10bbe21d1509/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.482777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9767ed3b35f84c65a1e8d8d1355154dd, user_id=224365c7065347d3820a10bbe21d1509, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.486456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45bef1-9a5f-4d74-bfe1-a75883c38077 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.497004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.499291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.516299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1035/2106] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:15 2022] PUT /identity/v3/projects/9767ed3b35f84c65a1e8d8d1355154dd/users/224365c7065347d3820a10bbe21d1509/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:15.518235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.518623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.518998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.527595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1072/2107] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:15.557860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] MFA Rules not processed for user `224365c7065347d3820a10bbe21d1509`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:15.559565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:15.654023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:15.695474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1036/2108] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:15.710646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:15.710646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:15.801501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da986c8-4f44-4c5c-b199-25f62e99a873 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:15.811856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:15.871354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:15.871966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:15.872243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:15.872645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:15.885136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:15.887969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:15.905903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1073/2109] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:16.259832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:16.261494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:16.402972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31267b2-54ee-4b54-b923-352d43cb618a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:16.436124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:16.437373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:16.438273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:16.440768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] PATH_INFO: `/v3/projects/b0e592d482f14552bce0f0fbd31e0c5a` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:16.442213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e592d482f14552bce0f0fbd31e0c5a)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:16.445290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:16.565462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e592d482f14552bce0f0fbd31e0c5a', 'request_id': 'req-5e568a81-e967-4987-b54f-0bb6b7b75fa5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.581018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e592d482f14552bce0f0fbd31e0c5a', 'request_id': 'req-5e568a81-e967-4987-b54f-0bb6b7b75fa5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.594514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e592d482f14552bce0f0fbd31e0c5a', 'request_id': 'req-5e568a81-e967-4987-b54f-0bb6b7b75fa5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.609395 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] The token cache is being invalidate because project b0e592d482f14552bce0f0fbd31e0c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:16.609697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.610388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.616077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1037/2110] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:16 2022] DELETE /identity/v3/projects/b0e592d482f14552bce0f0fbd31e0c5a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:16.617429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:16.654924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe40c1-3a93-4d97-84e0-18e75ef24951 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:16.759919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:16.760919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:16.761046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:16.761345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] PATH_INFO: `/v3/users/cf3c90263b9f4079ab5848b9cb307e9c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:16.762553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3c90263b9f4079ab5848b9cb307e9c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:16.764643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:16.838006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3c90263b9f4079ab5848b9cb307e9c'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.851541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3c90263b9f4079ab5848b9cb307e9c'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.874084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3c90263b9f4079ab5848b9cb307e9c'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.895939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3c90263b9f4079ab5848b9cb307e9c'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.898005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3c90263b9f4079ab5848b9cb307e9c'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:16.911560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1074/2111] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:16 2022] DELETE /identity/v3/users/cf3c90263b9f4079ab5848b9cb307e9c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:17.379001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:17.380629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.401046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568a81-e967-4987-b54f-0bb6b7b75fa5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:17.434420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:17.435819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.510572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:17.558945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a6d13-9c63-4599-9a18-b1870a8bcdf0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:17.565527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:17.566228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.566562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.566840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] PATH_INFO: `/v3/users/9b2872a9477e46a79e437da48103b5eb` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.567998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2872a9477e46a79e437da48103b5eb)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:17.571092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:17.573430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:17.574744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.575096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.581061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] PATH_INFO: `/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.582705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c42f211cb9a4448b3368d8fcfe0c3b5)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:17.586109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:17.617625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:17.645668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2872a9477e46a79e437da48103b5eb'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.660759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2872a9477e46a79e437da48103b5eb'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.683121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2872a9477e46a79e437da48103b5eb'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.683276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c42f211cb9a4448b3368d8fcfe0c3b5', 'request_id': 'req-e89aa534-387b-4087-964d-04ff2b780483'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.691299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c42f211cb9a4448b3368d8fcfe0c3b5', 'request_id': 'req-e89aa534-387b-4087-964d-04ff2b780483'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.694194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2872a9477e46a79e437da48103b5eb'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.695368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ba2a7e89-838b-492e-b5ed-14b04c2253dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2872a9477e46a79e437da48103b5eb'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.706047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1075/2112] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:17 2022] DELETE /identity/v3/users/9b2872a9477e46a79e437da48103b5eb => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:17.710958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.711435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.712047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.717547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c42f211cb9a4448b3368d8fcfe0c3b5', 'request_id': 'req-e89aa534-387b-4087-964d-04ff2b780483'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.735756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] The token cache is being invalidate because project 0c42f211cb9a4448b3368d8fcfe0c3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:17.736083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.736809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e89aa534-387b-4087-964d-04ff2b780483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:17.739750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:17.742465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1038/2113] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:17 2022] DELETE /identity/v3/projects/0c42f211cb9a4448b3368d8fcfe0c3b5 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:17.742631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.746072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98436-673b-4f6b-90c4-f3c81cdfd135 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.746359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98436-673b-4f6b-90c4-f3c81cdfd135 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.746722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc98436-673b-4f6b-90c4-f3c81cdfd135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.781118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-fbc98436-673b-4f6b-90c4-f3c81cdfd135 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:17.782246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fbc98436-673b-4f6b-90c4-f3c81cdfd135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.841161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1076/2114] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:17.854755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:17.856323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.880333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1039/2115] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:17.890176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.890464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.890789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.916560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:17.918416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:17.930491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1040/2116] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:17.937085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:17.942352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ae998-ab45-4d57-bc13-4c6424c59546 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:17.946527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73774d58-dbc1-4e13-9833-6c821d23c315 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:17.947609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:17.948323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.948739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.949534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.951337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:17.951935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:17.952213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:17.952483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:17.953160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:17.953456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:17.955048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ab27-b6e0-4e2a-9543-cd602e825388 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:17.960349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:17.961927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:17.966075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:17.978046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1077/2117] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:17.979057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1041/2118] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:17.983164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:17.994890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c99da-bcc2-43c6-8e43-01be42add555 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:17.999959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.001135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.001637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.001923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.002611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.005333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6101c08-ffb6-414c-942e-4b0305ed2ec8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.015678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4de714-ae8b-4293-9d52-d7ef7d582b33 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.017037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4de714-ae8b-4293-9d52-d7ef7d582b33 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.017374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4de714-ae8b-4293-9d52-d7ef7d582b33 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.022313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1042/2119] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:53:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:53:18.039244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1078/2120] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:17 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:18.043796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4de714-ae8b-4293-9d52-d7ef7d582b33 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.054268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.055269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.058100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.103064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:18.107492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.177947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4de714-ae8b-4293-9d52-d7ef7d582b33 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.183854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.184467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.184788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.185061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.185688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.187605 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.205334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1079/2121] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:18.206363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.208461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.212572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1043/2122] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:18.216548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.225166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db457081-4e56-41d8-be15-0d5c9420d9dc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.229118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636bcaf-10a7-4e80-a10a-2f6649897a70 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.230272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.230914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.231331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.231608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] PATH_INFO: `/v3/users/3a77211762af4cc9b4f0a2f74ac3ae1d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.232770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a77211762af4cc9b4f0a2f74ac3ae1d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.234619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.235227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.235388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.235492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.235763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.236550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.238219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.250662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1044/2123] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:18.251734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.253791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.263878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:18.269028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b54d-119a-4643-a177-ecb2bd2b1c43 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.278216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.279224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.279808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.280091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.289954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.291908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.295350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a77211762af4cc9b4f0a2f74ac3ae1d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.302912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1045/2124] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:18.303735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.304104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a77211762af4cc9b4f0a2f74ac3ae1d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.317155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c67f0-59e5-4799-ba4d-3901f77f858c service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.317938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a77211762af4cc9b4f0a2f74ac3ae1d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.324249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.324858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.325142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.325433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.326048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.327295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a77211762af4cc9b4f0a2f74ac3ae1d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.327647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.328242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a77211762af4cc9b4f0a2f74ac3ae1d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.340873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1080/2125] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:18 2022] DELETE /identity/v3/users/3a77211762af4cc9b4f0a2f74ac3ae1d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:18.341933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.343798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.359890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1046/2126] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:18.363984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.370661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.387223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4abd19-0061-46f7-9ff5-9376004aab62 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.410667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:18.434262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f84cb-c67a-41ed-9331-e75151a8467f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.438807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.439999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.440492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.440875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] PATH_INFO: `/v3/projects/e5c383f0136f4c828cd03c80df2bee75` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.442317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5c383f0136f4c828cd03c80df2bee75)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.445101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.473390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.474200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.474589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.474933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] PATH_INFO: `/v3/projects/b79cd429d1a44c3f8b008dbb8316d3b5/users/7198cda28f854b72b9fbf71b9a20bc9b/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.493289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79cd429d1a44c3f8b008dbb8316d3b5, user_id=7198cda28f854b72b9fbf71b9a20bc9b, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.496049 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7495789-65c3-46c1-a6a9-1cea1bd07cc4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.506809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5c383f0136f4c828cd03c80df2bee75', 'request_id': 'req-092b3969-babe-48e1-84cd-1c8fd04693dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.511977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1047/2127] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:18 2022] PUT /identity/v3/projects/b79cd429d1a44c3f8b008dbb8316d3b5/users/7198cda28f854b72b9fbf71b9a20bc9b/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:18.520946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5c383f0136f4c828cd03c80df2bee75', 'request_id': 'req-092b3969-babe-48e1-84cd-1c8fd04693dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.520946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5c383f0136f4c828cd03c80df2bee75', 'request_id': 'req-092b3969-babe-48e1-84cd-1c8fd04693dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.523874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.524281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.524644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.539966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] The token cache is being invalidate because project e5c383f0136f4c828cd03c80df2bee75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:18.540359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.541416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.545936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1081/2128] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:18 2022] DELETE /identity/v3/projects/e5c383f0136f4c828cd03c80df2bee75 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:18.547148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.548762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.566874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] MFA Rules not processed for user `7198cda28f854b72b9fbf71b9a20bc9b`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:18.568703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.635561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b3969-babe-48e1-84cd-1c8fd04693dc admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.638360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1048/2129] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:18.641822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.641822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.691964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.692515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:18.692697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.692890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.693683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] PATH_INFO: `/v3/users/b891e47a3959487580d682fc561bad98` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.694820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b891e47a3959487580d682fc561bad98)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.696784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.707942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec15f9-75a7-46ed-909f-8875b64ee502 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.713335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.713993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.714304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.714575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] PATH_INFO: `/v3/projects/72aa7ea3cb69454797bd7956e12f2a5e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.715688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72aa7ea3cb69454797bd7956e12f2a5e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.717677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.721446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:18.748394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b891e47a3959487580d682fc561bad98'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.755189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b891e47a3959487580d682fc561bad98'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.765552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72aa7ea3cb69454797bd7956e12f2a5e', 'request_id': 'req-bf789a47-0a6d-4b04-a55d-cb18486b28d4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.769452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b891e47a3959487580d682fc561bad98'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.771810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72aa7ea3cb69454797bd7956e12f2a5e', 'request_id': 'req-bf789a47-0a6d-4b04-a55d-cb18486b28d4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.778144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b891e47a3959487580d682fc561bad98'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.778998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b891e47a3959487580d682fc561bad98'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.780727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72aa7ea3cb69454797bd7956e12f2a5e', 'request_id': 'req-bf789a47-0a6d-4b04-a55d-cb18486b28d4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.785147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1082/2130] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:18 2022] DELETE /identity/v3/users/b891e47a3959487580d682fc561bad98 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:18.785905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.787259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.793616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] The token cache is being invalidate because project 72aa7ea3cb69454797bd7956e12f2a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:18.794644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.796100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bf789a47-0a6d-4b04-a55d-cb18486b28d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:18.798217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1049/2131] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:18 2022] DELETE /identity/v3/projects/72aa7ea3cb69454797bd7956e12f2a5e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:18.803106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.803532 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.803934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.827571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:18.829094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.863039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f43c8-38ad-4226-8cfc-a95d88b069a4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.869423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:18.870178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.870584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.870960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.871728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:18.874269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cbe58-f29d-45d2-a442-09aa9829d4d0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:18.904231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1050/2132] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:18.908234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1083/2133] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:18.909102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:18.910871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:18.911856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:18.912207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:18.912573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:18.926359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424054a-e2fb-4b48-9444-b70fe0989ee8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:18.934336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:18.935637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.032110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.033138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.033638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.034122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.034972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.036997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1084/2134] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:19.037133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.041329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.052077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391052b-58dd-4f52-9489-e4385fdd4742 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.057664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.058273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.058548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.059295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.059594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:19.059879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.061747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.081307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1051/2135] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:18 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:19.085338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.085532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1085/2136] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:19.091773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.097520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9f40c-ff20-41c6-b840-57a10faeea03 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.103416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453023d2-6886-47d4-8ac4-f4d153cfd75a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.108512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.109398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.109905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.110546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.111387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.111633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.112009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.112320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.112612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.114797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.116356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.119429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.131773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.135784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1052/2137] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:19.140139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.148717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.154977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e0003-3822-4c5c-b0f3-c36febb977c8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.160879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1086/2138] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:19.162795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.259954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a476aa3-ec49-4c3d-b471-8fdb1e148f18 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.264842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.265460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.265749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.266014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.266657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.268876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.272464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.273360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.273729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.274075 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] PATH_INFO: `/v3/projects/14fcb9bd0bc94f61aeea972c820351f4/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.289346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14fcb9bd0bc94f61aeea972c820351f4, user_id=c5a1ea2dfc584b25a4d721dfdc8e9d2e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.291472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1087/2139] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:19.292200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.292411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.294024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.306664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1053/2140] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:19 2022] PUT /identity/v3/projects/14fcb9bd0bc94f61aeea972c820351f4/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:19.309569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafee59a-7ce1-4d15-bf8d-ec71c44cbd9c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.309785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.366635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.367178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.367454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.367727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] PATH_INFO: `/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.368862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34fc0f49349b433cb6c4e6ad5a12f3b1)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.370781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.371820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1dbf-eefc-4e18-ac22-2ca2c22a5cd2 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.376670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.377273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.377554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.377822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.378461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.380166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.393648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1054/2141] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:19.393648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.405608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b02cd-1901-4a08-99c6-09b803b067a8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.411103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.412313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.412862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.413356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.414114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.415814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.423658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34fc0f49349b433cb6c4e6ad5a12f3b1', 'request_id': 'req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:19.426277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1055/2142] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:19.427164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.434721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34fc0f49349b433cb6c4e6ad5a12f3b1', 'request_id': 'req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:19.437351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c170a64-2d94-435e-9f42-aa81db85a6f7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.440996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34fc0f49349b433cb6c4e6ad5a12f3b1', 'request_id': 'req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:19.454535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] The token cache is being invalidate because project 34fc0f49349b433cb6c4e6ad5a12f3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:19.454823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:19.455452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:19.457494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1088/2143] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:19 2022] DELETE /identity/v3/projects/34fc0f49349b433cb6c4e6ad5a12f3b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:19.459264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.461426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.474146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.497988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.498610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.498883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.499154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.499971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.502687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d4c76-1cf9-457b-9734-61763f58e11a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.515822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1056/2144] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:19.517405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.517901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.518824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.536512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.540409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa7c2-40f6-414f-bbab-ad6ba08c0a6d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.543005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:19.544338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.545345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.546000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.546278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.546547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] PATH_INFO: `/v3/projects/14fcb9bd0bc94f61aeea972c820351f4/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.548414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=14fcb9bd0bc94f61aeea972c820351f4, user_id=c5a1ea2dfc584b25a4d721dfdc8e9d2e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.551167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fa204-b0b0-4cb1-b59d-cc8b73336aea admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.553627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1057/2145] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:19.554464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.555774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.562199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1089/2146] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:19 2022] PUT /identity/v3/projects/14fcb9bd0bc94f61aeea972c820351f4/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:19.563978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601127-948f-477c-978b-315764f05a2d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.564422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601127-948f-477c-978b-315764f05a2d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.564820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601127-948f-477c-978b-315764f05a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.573947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c36e6a-17c9-43a2-acb8-d06ba1fd7351 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.585871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d1601127-948f-477c-978b-315764f05a2d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:19.587067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d1601127-948f-477c-978b-315764f05a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.631176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.631730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.632010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.632286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] PATH_INFO: `/v3/projects/f64199a6685c4aa085da30f2ae3b679a/users/23d32c4aa2924ea5b82378be92eca85c/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.647887 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64199a6685c4aa085da30f2ae3b679a, user_id=23d32c4aa2924ea5b82378be92eca85c, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.650875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.651401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1090/2147] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:19.652837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.653180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.653526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.668217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1058/2148] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:19 2022] PUT /identity/v3/projects/f64199a6685c4aa085da30f2ae3b679a/users/23d32c4aa2924ea5b82378be92eca85c/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:19.669427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.701630 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] MFA Rules not processed for user `c5a1ea2dfc584b25a4d721dfdc8e9d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:19.702971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.758131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016cdc5-fc88-4a9e-b9e6-a9e91b316a7b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.767533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.768522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.768844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.769210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.770192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:19.770531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.772755 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281a0f4-863f-4f53-9b8a-f33c2d17134b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.789001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1059/2149] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:19.790441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.790786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.791120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.795531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1091/2150] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:19.796908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.810236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5693813-e28a-4cc7-b8c0-a8782da66453 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.816077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.816761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.821690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.822055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:19.822699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:19.824694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:19.832193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] MFA Rules not processed for user `23d32c4aa2924ea5b82378be92eca85c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:19.833481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.854870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1092/2151] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:19.856026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.860304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:19.872056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.922230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1060/2152] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:19.922917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:19.934098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260aad21-b304-49b2-8b4f-25a4101bdb5c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.935673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae4b4f-b9d6-4b58-a45f-2999028701c3 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:19.941121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:19.945637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:19.946575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:19.946985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:19.947357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.010155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.010885 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.011174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.011498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.012265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.014603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.029655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.031485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.043812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1093/2153] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:20.046963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.059367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3028-30e3-4be4-b7a3-b476cc757716 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.065770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1061/2154] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:19 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:20.071166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.071407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.072497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.073443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.073443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.074136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.077308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.091667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e65a8-c00b-487f-ab2c-67a02283fa00 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.102754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.103697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.104003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.104265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.105165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.107091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.120783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1062/2155] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:20.124148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.129216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1094/2156] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:20 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:20.131965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.147656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75066698-7e48-4a80-b81d-a26689f065aa admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.229948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.231243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.231587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.232193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.233097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.235077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.237376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2486fe5-2634-41da-93ec-dd5a69c366a1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.242680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.243558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.243844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.244115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.244793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.246890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.259871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1063/2157] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:20.262193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.281016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186709c0-2dea-4466-a795-96198b5c73c8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.286179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.287088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.287478 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.287848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] PATH_INFO: `/v3/projects/b93ff9132e1b4aab928b6cb013fb0693/users/1ed848b3a533404f84f9d85320a853aa/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.304068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1095/2158] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:20 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:20.305170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.306859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93ff9132e1b4aab928b6cb013fb0693, user_id=1ed848b3a533404f84f9d85320a853aa, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.313627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6fc52-fe6d-4a0b-8931-d9ebdd52a0f6 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.326746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46044dc7-ca07-43ed-8d23-71816e767956 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.334702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1064/2159] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:20 2022] PUT /identity/v3/projects/b93ff9132e1b4aab928b6cb013fb0693/users/1ed848b3a533404f84f9d85320a853aa/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:20.346067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.346426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.346791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.365382 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:20.386109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] MFA Rules not processed for user `1ed848b3a533404f84f9d85320a853aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:20.387711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:20.445088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.445994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.448530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.449927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.451053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.455083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.472520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:20.477677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1096/2160] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:20.479157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.492422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360f18c-056f-4179-9e12-27fba6f56816 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.497488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.498229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.498575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.498924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] PATH_INFO: `/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592/users/0b315b62a90648699ea7e44f89584424/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.515942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1065/2161] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:20.524467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fc70a92fb04f37afb4aa8c826a5592, user_id=0b315b62a90648699ea7e44f89584424, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.527441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.544454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.547338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:20.551484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1097/2162] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:20 2022] PUT /identity/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592/users/0b315b62a90648699ea7e44f89584424/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:20.557728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.623388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae425ef-86eb-4c9f-b5a0-2759dea0fa21 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.655598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cde2c-29ca-4c5d-8997-0dc2d41a264c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.661710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.662426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.662953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.663349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.663996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.665758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.681286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1098/2163] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:20.683057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.691741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.692645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.693854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.694285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.695068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60630ea6-37f8-40eb-891d-7af33ee62f68 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.700439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.701668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.702044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.702321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] PATH_INFO: `/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592/users/0b315b62a90648699ea7e44f89584424/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.704318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fc70a92fb04f37afb4aa8c826a5592, user_id=0b315b62a90648699ea7e44f89584424, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.707061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a415f2-bafd-45a8-bf85-6a23b9e3527e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.718170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1099/2164] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:20 2022] PUT /identity/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592/users/0b315b62a90648699ea7e44f89584424/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:20.731193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.731638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.731829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.764570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] MFA Rules not processed for user `0b315b62a90648699ea7e44f89584424`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:20.765540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:20.766366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.767936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.782931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1066/2165] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:20.817585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.835183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec3ff1-49ee-4ebb-a115-7f38087848c3 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.879799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1100/2166] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:20.908422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:20.916269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:20.921370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:20.925051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.926446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:20.926446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:20.926621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] PATH_INFO: `/v3/users/1ed848b3a533404f84f9d85320a853aa` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:20.928442 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed848b3a533404f84f9d85320a853aa)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:20.930601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:20.993599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d4ecd-8497-40ea-bd13-be160ec0d422 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:20.995048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed848b3a533404f84f9d85320a853aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:20.999288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:20.999927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.000211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.000485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] PATH_INFO: `/v3/users/7198cda28f854b72b9fbf71b9a20bc9b` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.001685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7198cda28f854b72b9fbf71b9a20bc9b)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.003607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.004648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed848b3a533404f84f9d85320a853aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.025640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed848b3a533404f84f9d85320a853aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.034880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed848b3a533404f84f9d85320a853aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.037688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed848b3a533404f84f9d85320a853aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.048697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1067/2167] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:20 2022] DELETE /identity/v3/users/1ed848b3a533404f84f9d85320a853aa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:21.049955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.051464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.066803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7198cda28f854b72b9fbf71b9a20bc9b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.075668 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7198cda28f854b72b9fbf71b9a20bc9b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.091207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7198cda28f854b72b9fbf71b9a20bc9b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.100563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7198cda28f854b72b9fbf71b9a20bc9b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.101289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-53fd98f9-9ca6-446d-9976-2a98a0aa1a93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7198cda28f854b72b9fbf71b9a20bc9b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.111440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1101/2168] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:20 2022] DELETE /identity/v3/users/7198cda28f854b72b9fbf71b9a20bc9b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:21.113132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88783714-2b15-41bd-ab90-52a2722cd518 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.113464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88783714-2b15-41bd-ab90-52a2722cd518 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.113739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88783714-2b15-41bd-ab90-52a2722cd518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.132708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f26ab-6175-49ca-bbea-1fafd6680b44 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:21.146471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-88783714-2b15-41bd-ab90-52a2722cd518 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:21.148000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-88783714-2b15-41bd-ab90-52a2722cd518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.201358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:21.202060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.202406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.202746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.204587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.227507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1102/2169] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:21.228847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.229353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.229480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.256386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:21.257077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:21.257652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.265848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1103/2170] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:21.266467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.267692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.280793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.282401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.290401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:21.292876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1068/2171] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:21.295722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.296753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.346861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf2493-e5c0-4c10-bc14-c61ba2e1d6ae None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:21.352479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:21.353237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.353660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.354119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.366898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.368812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.373702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe001bc0-7889-438b-a423-4e868eb38afb service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:21.386536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1104/2172] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:21.388229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:21.389474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.389955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.390492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.401082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.402768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.418300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1069/2173] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:21.718646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.720723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.745994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555171a-58a9-4c05-bb59-7f82ab975d74 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:21.755517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:21.756219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.756807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.757011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] PATH_INFO: `/v3/users/224365c7065347d3820a10bbe21d1509` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.758466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] RBAC: Authorizing `identity:delete_user(user_id=224365c7065347d3820a10bbe21d1509)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.760470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.828830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.832707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.838297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224365c7065347d3820a10bbe21d1509'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.851836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027a3df-35ab-46d6-be58-47da4b9087cb service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:21.861554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:21.862603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:21.862901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:21.863175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] PATH_INFO: `/v3/users/23d32c4aa2924ea5b82378be92eca85c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:21.865177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d32c4aa2924ea5b82378be92eca85c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:21.866404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224365c7065347d3820a10bbe21d1509'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.867777 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:21.909396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224365c7065347d3820a10bbe21d1509'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.928751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224365c7065347d3820a10bbe21d1509'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.930761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224365c7065347d3820a10bbe21d1509'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.939373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1105/2174] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:21 2022] DELETE /identity/v3/users/224365c7065347d3820a10bbe21d1509 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:21.940886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:21.942471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:21.976163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d32c4aa2924ea5b82378be92eca85c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:21.985068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d32c4aa2924ea5b82378be92eca85c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.006809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d32c4aa2924ea5b82378be92eca85c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.018322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d32c4aa2924ea5b82378be92eca85c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.019023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1e3bdafa-82cb-479e-995d-7f184b41b2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d32c4aa2924ea5b82378be92eca85c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.029235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1070/2175] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:21 2022] DELETE /identity/v3/users/23d32c4aa2924ea5b82378be92eca85c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:22.030922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.031210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.031620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.040981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75dcfe-364d-430b-bb59-0a6bf699961d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.052671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:22.053759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.091758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:22.118445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.119117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.119495 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.120237 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] PATH_INFO: `/v3/projects/b93ff9132e1b4aab928b6cb013fb0693` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.121476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93ff9132e1b4aab928b6cb013fb0693)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.124471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.137081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:22.154165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1071/2176] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:22.155231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.158318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.172303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdb37-c5f5-4a24-aa05-119bd7cf1680 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.177642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.178220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.178824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.179108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] PATH_INFO: `/v3/projects/b79cd429d1a44c3f8b008dbb8316d3b5` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.180223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79cd429d1a44c3f8b008dbb8316d3b5)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.182187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.194155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93ff9132e1b4aab928b6cb013fb0693', 'request_id': 'req-47581880-fc35-417e-9ece-cc756373bfd6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.203568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93ff9132e1b4aab928b6cb013fb0693', 'request_id': 'req-47581880-fc35-417e-9ece-cc756373bfd6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.212500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93ff9132e1b4aab928b6cb013fb0693', 'request_id': 'req-47581880-fc35-417e-9ece-cc756373bfd6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.231479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] The token cache is being invalidate because project b93ff9132e1b4aab928b6cb013fb0693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:22.231952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.232523 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-47581880-fc35-417e-9ece-cc756373bfd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.234505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1106/2177] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:21 2022] DELETE /identity/v3/projects/b93ff9132e1b4aab928b6cb013fb0693 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:22.235809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4395-20e2-473d-b58d-ae186d506b37 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.236099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4395-20e2-473d-b58d-ae186d506b37 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.236368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4395-20e2-473d-b58d-ae186d506b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.241947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79cd429d1a44c3f8b008dbb8316d3b5', 'request_id': 'req-829ab554-8d9f-47a4-92f0-16014f244315'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.256542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79cd429d1a44c3f8b008dbb8316d3b5', 'request_id': 'req-829ab554-8d9f-47a4-92f0-16014f244315'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.260967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-324d4395-20e2-473d-b58d-ae186d506b37 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:22.262586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-324d4395-20e2-473d-b58d-ae186d506b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.265173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79cd429d1a44c3f8b008dbb8316d3b5', 'request_id': 'req-829ab554-8d9f-47a4-92f0-16014f244315'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.278958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] The token cache is being invalidate because project b79cd429d1a44c3f8b008dbb8316d3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:22.279336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.280220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.284706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1072/2178] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:22 2022] DELETE /identity/v3/projects/b79cd429d1a44c3f8b008dbb8316d3b5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:22.285610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.287164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.345326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1107/2179] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:22.347061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.347359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.347638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.368214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ab554-8d9f-47a4-92f0-16014f244315 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.372478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:22.372693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.373150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.373578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.373870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.374002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.376371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.384214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1108/2180] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:22.385123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.386420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.408664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.408664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.421989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ceb1ec-debb-447b-9eec-a2c65422af4e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.427947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.428900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.429336 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.430636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] PATH_INFO: `/v3/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.432329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a1ea2dfc584b25a4d721dfdc8e9d2e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.435117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.437279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1073/2181] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:22.438598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.440137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.457300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c182074-48e7-41f8-9f72-d7b25e85d329 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.462440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.463309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.463795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.464303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.476145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.478057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70ff0c-fea2-44b5-a2ff-038b776f5098 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.490491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1074/2182] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:22.492573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.493130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.493628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.519008 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:22.521000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.532646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1075/2183] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:22.537699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.542097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a1ea2dfc584b25a4d721dfdc8e9d2e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.553297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a1ea2dfc584b25a4d721dfdc8e9d2e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.559700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:22.562015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ef916-70c1-4a61-a34a-51eb46319a7a None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.573066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.573706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.574223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.576082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.577734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:22.578405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.580368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.581385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a1ea2dfc584b25a4d721dfdc8e9d2e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.593356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a1ea2dfc584b25a4d721dfdc8e9d2e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.594334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a1ea2dfc584b25a4d721dfdc8e9d2e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:22.602652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1076/2184] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:22.604464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.605792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1109/2185] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:22 2022] DELETE /identity/v3/users/c5a1ea2dfc584b25a4d721dfdc8e9d2e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:22.610538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.625986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422f76-1a9e-4d2a-a187-e59dbcb6da4a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.695779 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:22.713861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.714632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.715238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.715371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.716714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.718860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.764158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1077/2186] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:22.768582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.786271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc7e28-0b7f-40db-8a31-857fe4d8743d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.857143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.859309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:22.890972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.891608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.891890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.892173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.892860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.901307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.940151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1110/2187] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:22 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:22.945352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.964577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7877e-8b66-48e2-aaa8-8d07a54fba85 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:22.972357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:22.973520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:22.974205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:22.974487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:22.975419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:22.977980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:22.993953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1111/2188] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:22.995121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:22.997179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.008847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e619c0-62c9-42fc-9e2b-36955a892e50 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.014977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.015636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.015919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.016195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] PATH_INFO: `/v3/projects/9767ed3b35f84c65a1e8d8d1355154dd` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.017396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9767ed3b35f84c65a1e8d8d1355154dd)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.019094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9890a3-eb3d-4894-b85c-3796880fa1fa admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.020833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.046914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.049119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.049412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.049688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] PATH_INFO: `/v3/projects/f64199a6685c4aa085da30f2ae3b679a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.051397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64199a6685c4aa085da30f2ae3b679a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.061310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.102496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9767ed3b35f84c65a1e8d8d1355154dd', 'request_id': 'req-ce10e29c-fc5b-4607-9896-d89723dc0afc'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.113056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9767ed3b35f84c65a1e8d8d1355154dd', 'request_id': 'req-ce10e29c-fc5b-4607-9896-d89723dc0afc'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.123812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9767ed3b35f84c65a1e8d8d1355154dd', 'request_id': 'req-ce10e29c-fc5b-4607-9896-d89723dc0afc'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.133502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64199a6685c4aa085da30f2ae3b679a', 'request_id': 'req-32998971-bf9f-487b-8eef-f534f8d361e3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.146878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] The token cache is being invalidate because project 9767ed3b35f84c65a1e8d8d1355154dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:23.147178 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.147362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64199a6685c4aa085da30f2ae3b679a', 'request_id': 'req-32998971-bf9f-487b-8eef-f534f8d361e3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.147894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.152884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1078/2189] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:22 2022] DELETE /identity/v3/projects/9767ed3b35f84c65a1e8d8d1355154dd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.152884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.153129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.160050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64199a6685c4aa085da30f2ae3b679a', 'request_id': 'req-32998971-bf9f-487b-8eef-f534f8d361e3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.181527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] The token cache is being invalidate because project f64199a6685c4aa085da30f2ae3b679a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:23.182161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.183046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.184892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1112/2190] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:22 2022] DELETE /identity/v3/projects/f64199a6685c4aa085da30f2ae3b679a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.185833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.187353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.241666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:23.250026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10e29c-fc5b-4607-9896-d89723dc0afc admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.259716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:23.268401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32998971-bf9f-487b-8eef-f534f8d361e3 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.274210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.274847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.275123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.275399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] PATH_INFO: `/v3/projects/14fcb9bd0bc94f61aeea972c820351f4` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.276509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=14fcb9bd0bc94f61aeea972c820351f4)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.278474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.312561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.313215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.313497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.313828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.314441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.316095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8c064-eb83-494b-8e4d-1f98fb9765e5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.331663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1079/2191] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:23.331861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14fcb9bd0bc94f61aeea972c820351f4', 'request_id': 'req-2028bf98-ded2-45a3-9cef-8761823df36d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.331979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.331979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.331979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.338447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14fcb9bd0bc94f61aeea972c820351f4', 'request_id': 'req-2028bf98-ded2-45a3-9cef-8761823df36d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.345057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14fcb9bd0bc94f61aeea972c820351f4', 'request_id': 'req-2028bf98-ded2-45a3-9cef-8761823df36d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.351759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:23.353101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.362620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] The token cache is being invalidate because project 14fcb9bd0bc94f61aeea972c820351f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:23.363362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.363808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.365518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1113/2192] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:23 2022] DELETE /identity/v3/projects/14fcb9bd0bc94f61aeea972c820351f4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.366447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.367760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.470866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028bf98-ded2-45a3-9cef-8761823df36d admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.477012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.477720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.478014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.478300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] PATH_INFO: `/v3/users/0b315b62a90648699ea7e44f89584424` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.479804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b315b62a90648699ea7e44f89584424)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.484712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1080/2193] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:23.484712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.485023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.485122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.503080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea05488-72de-4534-be9b-a370f52a95b7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.510853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.512216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.512644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.513052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] PATH_INFO: `/v3/projects/daec5c8157244e228263e82116888f96/users/ba20dd690f504b6d96eda957f634d763/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.533645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec5c8157244e228263e82116888f96, user_id=ba20dd690f504b6d96eda957f634d763, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.538027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb7cff-6b79-4187-8453-5da844359c32 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.570214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1081/2194] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:23 2022] PUT /identity/v3/projects/daec5c8157244e228263e82116888f96/users/ba20dd690f504b6d96eda957f634d763/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.572052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.572434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.572837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.600674 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:23.601918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.612639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b315b62a90648699ea7e44f89584424'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.622936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b315b62a90648699ea7e44f89584424'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.639210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:23.641169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b315b62a90648699ea7e44f89584424'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.652349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b315b62a90648699ea7e44f89584424'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.653625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f5883923-2a10-411f-989d-4a8ac640ddf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b315b62a90648699ea7e44f89584424'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.660946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1114/2195] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:23 2022] DELETE /identity/v3/users/0b315b62a90648699ea7e44f89584424 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.662427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.662703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.662985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.696378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1082/2196] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:23.701563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.704725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] MFA Rules not processed for user `ba20dd690f504b6d96eda957f634d763`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:23.706136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.717242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17607c-17ed-4f75-b242-b8019892f769 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.747278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:23.788798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.790037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.790037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.790037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.791132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:23.791462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.793951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.799946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1115/2197] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:23.800973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.803199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.815408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1083/2198] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:53:23.816525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.819524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.820995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba729b1-2110-44dd-bbb4-5e2d492d7288 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.827735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.828563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.829064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.829432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] PATH_INFO: `/v3/users/7cdab5749ce44e34b059d1386b9c7ba6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.830729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdab5749ce44e34b059d1386b9c7ba6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.832953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.901260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6b91f-2348-48b6-a57f-3a044b0ce6dd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.903913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdab5749ce44e34b059d1386b9c7ba6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.905557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.909148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.909586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.909982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.916033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdab5749ce44e34b059d1386b9c7ba6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.920276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.921936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.932620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1084/2199] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:23.933446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:23.936705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdab5749ce44e34b059d1386b9c7ba6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.948901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdab5749ce44e34b059d1386b9c7ba6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.950625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-64d0d49d-38d2-481c-ad85-793827a1c77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdab5749ce44e34b059d1386b9c7ba6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:23.953786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0b0d-a873-4fad-a549-cff429321126 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:23.959294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:23.959897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.960184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.963187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.963187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:23.963187 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:23.963765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1116/2200] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:23 2022] DELETE /identity/v3/users/7cdab5749ce44e34b059d1386b9c7ba6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:23.965247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:23.965702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:23.965988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:23.992576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:23.993810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:23.995413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1085/2201] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:24.000240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.002955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.065467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:24.082584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1117/2202] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:24.084285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.084423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107576b-51f2-437c-b5e0-c4422ca9854f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.092029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.092029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.092029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.092029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.092432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.093565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.093945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.110729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14445f49-4821-4ae0-8b8a-539022577899 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.118646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.119460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.119855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.120250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] PATH_INFO: `/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.121880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fc70a92fb04f37afb4aa8c826a5592)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.124102 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.124823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1086/2203] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:23 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:24.125705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.127519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.180910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:24.190645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fc70a92fb04f37afb4aa8c826a5592', 'request_id': 'req-023a04d8-8d4d-4e68-9b1c-332d71237cc2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.196909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fc70a92fb04f37afb4aa8c826a5592', 'request_id': 'req-023a04d8-8d4d-4e68-9b1c-332d71237cc2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.204136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fc70a92fb04f37afb4aa8c826a5592', 'request_id': 'req-023a04d8-8d4d-4e68-9b1c-332d71237cc2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.209930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a465e5-39fb-4d8e-9f6f-3fe6533fc180 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.221503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] The token cache is being invalidate because project c1fc70a92fb04f37afb4aa8c826a5592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:24.229932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.230292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.232575 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1118/2204] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:24 2022] DELETE /identity/v3/projects/c1fc70a92fb04f37afb4aa8c826a5592 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:24.233648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.236146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.318816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.319993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.320673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.321145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.324261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.342195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a04d8-8d4d-4e68-9b1c-332d71237cc2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.347313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.348229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.348492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.348899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.350340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.352420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ad6bc-55ce-42d6-8b3f-d1bbe173f374 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.366662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1119/2205] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:24.366662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.366662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.366662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.390584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:24.393132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.403397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1120/2206] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:24.404110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.419954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b6f51-f98c-4f69-b292-9300351d80af None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.424854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.425545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.425896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.426241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.426957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.429680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0fad5-5ace-4382-b2fe-c37162087014 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.443718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.446377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.447487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1121/2207] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:24.449208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.449608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.449982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.461390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1087/2208] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:24.462541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.472460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:24.474122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.475467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fe317-2a9d-4c0b-958b-abb4601bdf7a service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.480641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.481609 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.481893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.482175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] PATH_INFO: `/v3/projects/d79626e4fcef49c784aeb5fdad057fe5/users/38c3e6abc0034f8d95bdcc9b8db71526/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.483775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1122/2209] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:24.487906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.497338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958a58-f2f5-4a3d-9db8-8ba71dbbb266 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.500512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79626e4fcef49c784aeb5fdad057fe5, user_id=38c3e6abc0034f8d95bdcc9b8db71526, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.502283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.503265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.503265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.503472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.503656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.504345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:24.504736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.506921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ee03e-dd55-4e83-9185-a9e09d5355a5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.519078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1088/2210] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:24 2022] PUT /identity/v3/projects/d79626e4fcef49c784aeb5fdad057fe5/users/38c3e6abc0034f8d95bdcc9b8db71526/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:24.524497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1123/2211] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:53:24.525678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.532210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.532491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.532793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.562921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] MFA Rules not processed for user `38c3e6abc0034f8d95bdcc9b8db71526`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:24.564145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.587043 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:24.627973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a36ea7-db0a-4e1c-80c8-622a2fbbbb31 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.637825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.638512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.638800 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.639076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.640698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.643819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.651383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1124/2212] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:24.652489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.669736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1320df-87ba-4085-91fe-1b32826a4668 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.673473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:24.683906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.684558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.684864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.685137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.685780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.687100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1089/2213] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:24.687813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.690758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.719820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1125/2214] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:24.723759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.818865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbac20-d6b8-4841-9db0-5dc94e3bf1fa admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.820133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a9f38-b2fe-4b4c-aa88-172006094170 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.826146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.826479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.826739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.827014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.827758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.829328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.830088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.830379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.830804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.831234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.832041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.836431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.868119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1090/2215] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:24.868119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.869191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:24.873370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1126/2216] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:24 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:24.875953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.885383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7884ba6-c106-46ae-bca4-f18ec7137383 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.890358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.891018 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.891137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f3db6-decd-4a41-a4d2-df7a48d416af admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.891299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.891564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] PATH_INFO: `/v3/projects/1d4601eb8dc141519a3d478cc5439e32` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.892706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4601eb8dc141519a3d478cc5439e32)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.894619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.896832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.897435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.897757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.898236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.899250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.902905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.914814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1127/2217] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:24.915752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.928533 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa324d-1e95-4fce-8c8d-63bf7283cd1f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.936753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:24.937403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:24.937748 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:24.938067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:24.938714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:24.940581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:24.954735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1128/2218] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:24.954735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.955883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4601eb8dc141519a3d478cc5439e32', 'request_id': 'req-538728b4-8163-4635-a281-14d2ee05f8e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.963194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504faca-af02-4311-aa57-df0892092594 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:24.965805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4601eb8dc141519a3d478cc5439e32', 'request_id': 'req-538728b4-8163-4635-a281-14d2ee05f8e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.974093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4601eb8dc141519a3d478cc5439e32', 'request_id': 'req-538728b4-8163-4635-a281-14d2ee05f8e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.987195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] The token cache is being invalidate because project 1d4601eb8dc141519a3d478cc5439e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:24.987488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.988149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:24.989873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1091/2219] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:24 2022] DELETE /identity/v3/projects/1d4601eb8dc141519a3d478cc5439e32 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:24.990640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:24.992109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.028709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.029635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.029910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.030178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.030824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.032426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.043811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1129/2220] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:25.047751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.049464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.059601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:25.060304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538728b4-8163-4635-a281-14d2ee05f8e8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.063841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9030e7-1b81-46c1-908d-15dd159b7e8e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.065433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.066046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.066362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.066678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.067421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.069117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.070814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.071446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.071729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.072006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] PATH_INFO: `/v3/users/89de26a87560404883e4affceb13a69d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.073180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89de26a87560404883e4affceb13a69d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.075087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.082211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1092/2221] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:25.082211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.082211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.094901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf719612-a005-4017-a5e3-342d8f71b078 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.100147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.100819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.101103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.101375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] PATH_INFO: `/v3/projects/b2cac3cfcbea45d1b253b1c5d566bed9/users/e1ba955e328a40318a36056695c6fc03/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.103602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:25.123894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2cac3cfcbea45d1b253b1c5d566bed9, user_id=e1ba955e328a40318a36056695c6fc03, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.126763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.132721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89de26a87560404883e4affceb13a69d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.140233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1093/2222] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:25 2022] PUT /identity/v3/projects/b2cac3cfcbea45d1b253b1c5d566bed9/users/e1ba955e328a40318a36056695c6fc03/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:25.140541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89de26a87560404883e4affceb13a69d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.141275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.158939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89de26a87560404883e4affceb13a69d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.167432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89de26a87560404883e4affceb13a69d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.168109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-e981714a-2dde-44c6-9c7e-f4a8acd6a100 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89de26a87560404883e4affceb13a69d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.174350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1130/2223] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:25 2022] DELETE /identity/v3/users/89de26a87560404883e4affceb13a69d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:25.175561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.175847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.176118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.199149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:25.200375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.232895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc56fc-28cf-4912-a312-fd0dee3d44d4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.278629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1131/2224] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:25.289834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.293075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.314704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.315418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.315703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.315984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] PATH_INFO: `/v3/projects/016775e86a454cf091353c8232df754c/users/162e9aed91ca49b388b7370d3a4ab99e/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.334416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=016775e86a454cf091353c8232df754c, user_id=162e9aed91ca49b388b7370d3a4ab99e, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.338372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcb619-2a49-4645-9e4c-00b66ff4f50f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.363199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1094/2225] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:25 2022] PUT /identity/v3/projects/016775e86a454cf091353c8232df754c/users/162e9aed91ca49b388b7370d3a4ab99e/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:25.374248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.374938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.374938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.377410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f808e0c-95ab-41d7-a8ee-84d0d262abcc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.413486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] MFA Rules not processed for user `162e9aed91ca49b388b7370d3a4ab99e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:25.414704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.428892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:25.466217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.467401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.467780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.468193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.513297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1095/2226] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:25.522899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.530209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.544473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:25.608105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.609866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.625343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1132/2227] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:25.626588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.628530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.662832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027afdc-5c50-4af2-891e-206f7053f575 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.667190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a261e-9661-41e2-b4f0-ac35c019b614 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.673220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.674373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.674877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.675539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.676547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.680616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.681112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.682065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.682346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.682616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] PATH_INFO: `/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.684685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0959ac2b4f45a0aafa1eb32003009d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.686862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.735607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1096/2228] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:25.743543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.801690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0959ac2b4f45a0aafa1eb32003009d', 'request_id': 'req-36cf527d-b55c-44f6-a462-b265c46dd668'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.811645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0959ac2b4f45a0aafa1eb32003009d', 'request_id': 'req-36cf527d-b55c-44f6-a462-b265c46dd668'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.819354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0959ac2b4f45a0aafa1eb32003009d', 'request_id': 'req-36cf527d-b55c-44f6-a462-b265c46dd668'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.834131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] The token cache is being invalidate because project 2a0959ac2b4f45a0aafa1eb32003009d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:25.834401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.835195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-36cf527d-b55c-44f6-a462-b265c46dd668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:25.835420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e8bf7-b9dc-4257-ab62-a5771d9adb2b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.836935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1133/2229] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:25 2022] DELETE /identity/v3/projects/2a0959ac2b4f45a0aafa1eb32003009d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:25.841653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b99e-14cc-4056-bc6b-18e5ff0e38e7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.841938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b99e-14cc-4056-bc6b-18e5ff0e38e7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.842218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b99e-14cc-4056-bc6b-18e5ff0e38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.866641 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-44c0b99e-14cc-4056-bc6b-18e5ff0e38e7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:25.868342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-44c0b99e-14cc-4056-bc6b-18e5ff0e38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.924025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.924803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.925081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.925353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.926505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.928926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.960881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1097/2230] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:25.961545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1134/2231] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:25.966812 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:25.969889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.970183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.970467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.971995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:25.986289 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e3c0c-1b5f-4995-a921-f3033b954d52 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:25.989990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:25.992635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:25.993339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:25.993691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:25.994042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:25.994760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:25.996564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:25.997497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:26.001640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.013191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1098/2232] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:26.016893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.017501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1135/2233] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:26.022209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.029078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de4117-b4ce-4bb4-9146-41932aa483e3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.032025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bfe8b-85df-467a-a761-782f2fafc62f None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.035459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.037218 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.037993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.038499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] PATH_INFO: `/v3/projects/c84b3ef5482d47499a263933eaabc4a1/users/910a18fcd7534538b164e81eecdc04d7/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.040053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.040826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.041173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.041505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.042400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:26.042777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.045252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.051814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:26.062396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1136/2234] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:26.066513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.082297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84b3ef5482d47499a263933eaabc4a1, user_id=910a18fcd7534538b164e81eecdc04d7, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.082650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0631e-9671-4ebe-972c-48fcfe2afabc admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.087859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.088909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.089675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.089966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.090361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.091080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.093174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.108074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1099/2235] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:26 2022] PUT /identity/v3/projects/c84b3ef5482d47499a263933eaabc4a1/users/910a18fcd7534538b164e81eecdc04d7/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:26.113557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.137173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1137/2236] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:26.141026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.207722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277c546-be1c-4f36-94e7-209c6c5e53eb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.233466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c374-6f40-4419-93a7-5bb48b2bc8be admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.239732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.240417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.240863 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.244266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.244266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.248100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.274963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1138/2237] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:26.279451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.284083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.284729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.285012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.285285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.286206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.288193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.290262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb9f49-5598-4f4a-b97f-708fbf8298da admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.295602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.296127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.296787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.296787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.297347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.300653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1100/2238] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:26.300786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.303206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.314299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1139/2239] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:26.316863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a17c4-e5c6-470d-b4aa-ca556353e34a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.318478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.322120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.322935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.323383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.323383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] PATH_INFO: `/v3/projects/c84b3ef5482d47499a263933eaabc4a1/users/910a18fcd7534538b164e81eecdc04d7/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.325408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84b3ef5482d47499a263933eaabc4a1, user_id=910a18fcd7534538b164e81eecdc04d7, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.328141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc834c6-8342-47e0-b27c-01cc11febfc7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.329968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670c5ac-7bd6-417d-b98e-68a4836b096b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.338012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.339416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.339704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.339977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.342182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.344224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.348930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1101/2240] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:26 2022] PUT /identity/v3/projects/c84b3ef5482d47499a263933eaabc4a1/users/910a18fcd7534538b164e81eecdc04d7/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:26.359130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23774-9be1-410d-89e6-5bede02801fb None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.359414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23774-9be1-410d-89e6-5bede02801fb None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.360261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23774-9be1-410d-89e6-5bede02801fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.361433 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1140/2241] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:26.366248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.379560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13b651-0b84-491b-981c-9efd914397ea admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.396999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-51b23774-9be1-410d-89e6-5bede02801fb None None] MFA Rules not processed for user `910a18fcd7534538b164e81eecdc04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:26.398513 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-51b23774-9be1-410d-89e6-5bede02801fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.471270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.471959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.472239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.473135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] PATH_INFO: `/v3/projects/a7f5208a751c47e2996d0958d46d555f/users/c251dd7e3bbe4c7497267e168ef8782a/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.488637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f5208a751c47e2996d0958d46d555f, user_id=c251dd7e3bbe4c7497267e168ef8782a, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.491909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1b7ed-6547-42d1-8ded-48c527546169 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.518568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1141/2242] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:26 2022] PUT /identity/v3/projects/a7f5208a751c47e2996d0958d46d555f/users/c251dd7e3bbe4c7497267e168ef8782a/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:26.524717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1102/2243] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:26.524843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.524843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.524843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.530608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.531947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.532240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.546236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:26.562185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] MFA Rules not processed for user `e1ba955e328a40318a36056695c6fc03`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:26.563423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.568292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:26.576869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] MFA Rules not processed for user `c251dd7e3bbe4c7497267e168ef8782a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:26.578141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.646140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1142/2244] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:26.647478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.649405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.649860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1103/2245] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:26.653959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.720722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26ef60-b74f-4e2f-87eb-9b9aab98d573 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.725188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.725913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.726254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.726620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.728551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.734753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c480b-3797-4b06-a6b1-427d50d174c2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.740322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.741200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.741365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.742327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.743060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.746626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.803241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1104/2246] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:26.803241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.811538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:26.825048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.826811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.841018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1143/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:26.842305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.868824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b924-fb58-4c63-8af4-1c809bee52ba service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.895155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb34d0e-afbf-48bb-9af5-7843543e156f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:26.901039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.901727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.901929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.902205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.902840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.904793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.941241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1105/2248] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:26.942290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:26.948172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:26.956670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:26.957574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:26.958028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:26.958493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:26.959909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:26.962048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:26.974778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:26.997184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1144/2249] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:26.998738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.026567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fccf96-b0d9-4e5e-ba49-b405c60b81c1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.032968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.034415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.034696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.034980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.035784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.038520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.038976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3e369-5408-4dde-916e-915993639420 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.045703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.046312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.046595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.047014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.048449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.049895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.051484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1145/2250] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:27.056698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.069496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf233c-0220-4251-a6b8-dcb8554ed3b8 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.072315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1106/2251] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:26 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:27.076242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.076734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.077398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.077721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.078195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] PATH_INFO: `/v3/projects/012da95a996a4fe290c5a43865ff5ede/users/d50c73a7820b47b1b442ac83127d2aa6/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.093680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57731f9d-df5a-4fe3-be7c-4660c789d33f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.100085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.100614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012da95a996a4fe290c5a43865ff5ede, user_id=d50c73a7820b47b1b442ac83127d2aa6, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.100739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.101138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.101419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.102066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.103656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.103788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.119296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1107/2252] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:27.119519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1146/2253] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:27 2022] PUT /identity/v3/projects/012da95a996a4fe290c5a43865ff5ede/users/d50c73a7820b47b1b442ac83127d2aa6/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:27.120836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.123095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.136282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549483b-2e37-428d-8eeb-e6529c52eb98 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.211613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.212293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.212826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.213357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] PATH_INFO: `/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2/users/5c8115ec2055401094f5a06f3e89672f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.223625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc76881-03fb-4578-8645-53ecaab85051 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.231767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.232388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.232566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852321a8a8e84ca1ab2d9fd7bff868a2, user_id=5c8115ec2055401094f5a06f3e89672f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.232907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.233188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.234019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.235523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.239981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.258818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1147/2254] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:27.260294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1108/2255] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:27 2022] PUT /identity/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2/users/5c8115ec2055401094f5a06f3e89672f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:27.264681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.264955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.276362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b84b6-9617-46a6-a662-71c97d3a98a3 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.342219 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.343045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.343311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.343581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] PATH_INFO: `/v3/projects/012da95a996a4fe290c5a43865ff5ede/users/d50c73a7820b47b1b442ac83127d2aa6/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.344999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969ab8-5d1e-429f-a851-c892c17c8259 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.345502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012da95a996a4fe290c5a43865ff5ede, user_id=d50c73a7820b47b1b442ac83127d2aa6, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.348138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.349923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.350525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.350807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.351268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.351780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.353603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbc5e2-1c3f-4025-bdfb-6c1a342db3b3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.358253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1148/2256] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:27 2022] PUT /identity/v3/projects/012da95a996a4fe290c5a43865ff5ede/users/d50c73a7820b47b1b442ac83127d2aa6/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:27.363660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1109/2257] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:27.368492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.368872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f11cf-cf5d-4686-bde5-4a21dc1c12db None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.369161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f11cf-cf5d-4686-bde5-4a21dc1c12db None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.369439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f11cf-cf5d-4686-bde5-4a21dc1c12db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.376400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:27.399457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7d3f11cf-cf5d-4686-bde5-4a21dc1c12db None None] MFA Rules not processed for user `d50c73a7820b47b1b442ac83127d2aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:27.400788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7d3f11cf-cf5d-4686-bde5-4a21dc1c12db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:27.414503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:27.431927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d6e1a-d0ca-46a1-ab50-d1373620aa65 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.436682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.437339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.437618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.437890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] PATH_INFO: `/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2/users/5c8115ec2055401094f5a06f3e89672f/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.439818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=852321a8a8e84ca1ab2d9fd7bff868a2, user_id=5c8115ec2055401094f5a06f3e89672f, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.442476 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.453202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1149/2258] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:27 2022] PUT /identity/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2/users/5c8115ec2055401094f5a06f3e89672f/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:27.469379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1110/2259] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:27.481332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:27.540008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315667c-fef2-4bd6-951c-db108384f94e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:27.543911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:27.544515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:27.544815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:27.545087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:27.607884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:27.609504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:27.620146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1150/2260] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:28.139823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.140272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.140533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.189834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] MFA Rules not processed for user `5c8115ec2055401094f5a06f3e89672f`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:28.191047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:28.269061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:28.283141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1111/2261] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:28.298959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:28.300546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:28.318279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b347f1-bba8-494e-a309-58b797fc0483 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:28.325034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:28.326140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.327524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.327988 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.358583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f62bd-ea76-406e-b154-1a9e01950066 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:28.363211 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:28.363946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.364295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.365281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] PATH_INFO: `/v3/users/c251dd7e3bbe4c7497267e168ef8782a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.366489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c251dd7e3bbe4c7497267e168ef8782a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:28.368500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:28.389288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:28.390728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab61b1-e55c-4281-8d36-c9f76c69b1cd service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:28.400965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1112/2262] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:28.402815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:28.418805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c251dd7e3bbe4c7497267e168ef8782a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:28.425659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c251dd7e3bbe4c7497267e168ef8782a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:28.438353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c251dd7e3bbe4c7497267e168ef8782a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:28.447294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c251dd7e3bbe4c7497267e168ef8782a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:28.447996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c251dd7e3bbe4c7497267e168ef8782a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:28.455152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1151/2263] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:28 2022] DELETE /identity/v3/users/c251dd7e3bbe4c7497267e168ef8782a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:28.459656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.459968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.460248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.480415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:28.481707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:28.486999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:28.557776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1113/2264] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:28.568255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:28.569998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:28.631827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b45548-3123-4e40-9551-c5b9de64174c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:28.635659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:28.636255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.636532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.636831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.647842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:28.649318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:28.660856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1152/2265] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:28.932497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:28.934319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:28.947338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2e076-7a4d-4295-9b1e-1169bf5b77ab None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:28.952635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:28.953346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:28.953689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:28.954025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] PATH_INFO: `/v3/projects/a7f5208a751c47e2996d0958d46d555f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:28.955249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f5208a751c47e2996d0958d46d555f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:28.957330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.007767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f5208a751c47e2996d0958d46d555f', 'request_id': 'req-88234ba9-4b0f-48eb-ad11-cf771138a999'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.013337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f5208a751c47e2996d0958d46d555f', 'request_id': 'req-88234ba9-4b0f-48eb-ad11-cf771138a999'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.021577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f5208a751c47e2996d0958d46d555f', 'request_id': 'req-88234ba9-4b0f-48eb-ad11-cf771138a999'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.032476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] The token cache is being invalidate because project a7f5208a751c47e2996d0958d46d555f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:29.032790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.033413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.034890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1114/2266] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:28 2022] DELETE /identity/v3/projects/a7f5208a751c47e2996d0958d46d555f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:29.039138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.048454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f826c27-a991-48cb-bbdb-99932b73394c service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.102855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.103514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.103792 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.104067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] PATH_INFO: `/v3/users/d50c73a7820b47b1b442ac83127d2aa6` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.105187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50c73a7820b47b1b442ac83127d2aa6)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.107187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.152899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50c73a7820b47b1b442ac83127d2aa6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.159804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:29.163777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50c73a7820b47b1b442ac83127d2aa6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.177356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50c73a7820b47b1b442ac83127d2aa6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.185090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50c73a7820b47b1b442ac83127d2aa6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.185746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-068f2695-46a8-46cb-b3ca-f16ee5f97301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50c73a7820b47b1b442ac83127d2aa6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.191703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1153/2267] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:29 2022] DELETE /identity/v3/users/d50c73a7820b47b1b442ac83127d2aa6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:29.507032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.508564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:29.523400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88234ba9-4b0f-48eb-ad11-cf771138a999 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.580948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.581704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.582146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.582463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] PATH_INFO: `/v3/projects/012da95a996a4fe290c5a43865ff5ede` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.584164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012da95a996a4fe290c5a43865ff5ede)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.586208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.622545 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:29.632222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012da95a996a4fe290c5a43865ff5ede', 'request_id': 'req-619c8e14-9d21-4449-a719-9644e390bb63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.638417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012da95a996a4fe290c5a43865ff5ede', 'request_id': 'req-619c8e14-9d21-4449-a719-9644e390bb63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.645394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012da95a996a4fe290c5a43865ff5ede', 'request_id': 'req-619c8e14-9d21-4449-a719-9644e390bb63'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.656938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] The token cache is being invalidate because project 012da95a996a4fe290c5a43865ff5ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:29.657221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.657835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-619c8e14-9d21-4449-a719-9644e390bb63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:29.659288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1115/2268] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:29 2022] DELETE /identity/v3/projects/012da95a996a4fe290c5a43865ff5ede => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:29.664726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.665214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.665308 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.684702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:29.685904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:29.748663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1154/2269] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:29.771693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.771693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.771693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.780908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:29.782137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:29.782497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.783924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:29.793459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1116/2270] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:29.798594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.807959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21dc35-7bdf-43cf-aba3-8bb0f0d1a611 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.812659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.813287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.813566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.813837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.814481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:29.814784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.817092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.836934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1117/2271] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:29.837953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.847775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795e229-a9e6-4456-8428-81bfd6c4e6c5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.855932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33c394-448a-4ade-a0b2-b504b5d239bd None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.856881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.858378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.858378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.858378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.858933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.859490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.860171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.860517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.860870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.861215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.862727 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:29.899603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1118/2272] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:29 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:29.904363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:29.963039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.964478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:29.977082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1155/2273] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:29.980451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a384dee-96e1-4249-9ca2-1474109ef1f8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:29.985799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:29.986426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:29.986705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:29.986974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:29.987604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:29.990190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.023315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1119/2274] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:29 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:30.024954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.040658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b03da7-f786-4e2b-bb57-9d61de832978 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.046525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.047250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.047625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.047999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.048797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.050949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.070809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1156/2275] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:30.071952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.086067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467ec9c-e86f-4f87-9076-7ce2219b3d85 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.095695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.095695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.095695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.095695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.095972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.095972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a2df6-c53c-4d33-a412-e0b3a41b4bf1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.113748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1120/2276] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:30.118029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.128397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a38d7-25e6-4d7d-89ab-07d34917f124 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.133377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.134040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.134319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.134586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] PATH_INFO: `/v3/projects/01f245e72ca84936ae2498f45ce267c3/users/ca94c18048dd46e3a00fbf01a4f0debf/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.140348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:30.156194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f245e72ca84936ae2498f45ce267c3, user_id=ca94c18048dd46e3a00fbf01a4f0debf, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.159546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.173545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1157/2277] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:30 2022] PUT /identity/v3/projects/01f245e72ca84936ae2498f45ce267c3/users/ca94c18048dd46e3a00fbf01a4f0debf/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:30.187089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.187089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.187089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.216497 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] MFA Rules not processed for user `ca94c18048dd46e3a00fbf01a4f0debf`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:30.217818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:30.273659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:30.294598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1121/2278] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:30.299260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.364258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3508d-b67b-4240-9ddd-aeace1c5d060 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.369579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.370190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.370474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.370749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.371390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.373394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.406316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1158/2279] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:30 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:30.410473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.423793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f645d-726e-4383-bcec-5a4f5304238c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.493446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.494206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.494490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.494827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.495856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.497971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.523794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1122/2280] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:30 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:30.528171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.540193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502ab7a-78ea-4a39-ad9c-817ccf46e8a9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.545691 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.546488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.546826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.547186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.549489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.551229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.563664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1159/2281] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:30.568037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.579780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4a76-7358-4b26-9f2f-0813aa0cc700 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.584873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.585550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.585829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.586103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] PATH_INFO: `/v3/projects/c52140ea5b1246d2939f12b55726cd01/users/8db4a7f0f6ea4f77935673856cb5f8f7/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.608301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52140ea5b1246d2939f12b55726cd01, user_id=8db4a7f0f6ea4f77935673856cb5f8f7, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.609854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.618604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.620700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:30.629389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1123/2282] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:30 2022] PUT /identity/v3/projects/c52140ea5b1246d2939f12b55726cd01/users/8db4a7f0f6ea4f77935673856cb5f8f7/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:30.633506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.697773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adfcb4-8cf0-43ae-a57e-a9f4e8f4300f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.709487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f2b9-7aeb-4b84-8e36-adb9555d9fb4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.714486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.715215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.715649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.716017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.716796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.718512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.729480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1124/2283] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:53:30.733561 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:30.749499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285aea96-d0d1-44f8-9612-447e50bcd633 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:30.754897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.755756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.756184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.756669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] PATH_INFO: `/v3/projects/c52140ea5b1246d2939f12b55726cd01/users/8db4a7f0f6ea4f77935673856cb5f8f7/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.758170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:30.758592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52140ea5b1246d2939f12b55726cd01, user_id=8db4a7f0f6ea4f77935673856cb5f8f7, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.759454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.759454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.759454 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.761705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6700446-d2ac-416d-ae03-208baa5677fc admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.761843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:30.772552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1125/2284] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:30 2022] PUT /identity/v3/projects/c52140ea5b1246d2939f12b55726cd01/users/8db4a7f0f6ea4f77935673856cb5f8f7/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:30.783654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:30.783944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:30.784229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:30.802554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:30.817682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] MFA Rules not processed for user `8db4a7f0f6ea4f77935673856cb5f8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:30.818912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:30.845261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:30.846777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:30.857330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:30.860638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1160/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:30.912809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1126/2286] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:31.153506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:31.162729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e2f8b-bf07-4028-8928-44ae0982e6a1 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:31.213977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:31.214504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:31.214781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:31.215047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:31.273215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:31.274649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:31.283350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1161/2287] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:32.399011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:32.408309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6b0d7-83ce-4cde-b14c-10f972c54f05 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:32.411731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:32.412317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:32.412595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:32.412897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:32.414589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:32.474703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:32.476126 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9f106-f3c8-4b8e-b3f4-a6096d08700a service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:32.485330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1127/2288] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:35.012972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:35.124288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b0dec-74b7-49e3-b589-c18cb70db58e service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:35.140963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:35.142201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:35.142636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:35.143244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] PATH_INFO: `/v3/users/ca94c18048dd46e3a00fbf01a4f0debf` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:35.144538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca94c18048dd46e3a00fbf01a4f0debf)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:35.147717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:35.190850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:35.234311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca94c18048dd46e3a00fbf01a4f0debf'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.248551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca94c18048dd46e3a00fbf01a4f0debf'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.267261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca94c18048dd46e3a00fbf01a4f0debf'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.290437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca94c18048dd46e3a00fbf01a4f0debf'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.291270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca94c18048dd46e3a00fbf01a4f0debf'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.305062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1162/2289] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:35 2022] DELETE /identity/v3/users/ca94c18048dd46e3a00fbf01a4f0debf => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:35.310343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:35.310822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:35.310944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:35.331529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:35.333311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:35.407422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1128/2290] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:35.417883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:35.419326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:35.480826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d80211-1694-4671-a69e-1fb5da14fb42 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:35.487119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:35.488172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:35.488694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:35.489175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:35.503426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:35.505749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:35.521205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1163/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:35.812383 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:35.816307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:35.844893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04344b8-ed20-425d-8a9b-90707573d6dc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:35.857540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:35.858335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:35.859047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:35.859331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] PATH_INFO: `/v3/projects/01f245e72ca84936ae2498f45ce267c3` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:35.860985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f245e72ca84936ae2498f45ce267c3)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:35.862995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:35.921843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f245e72ca84936ae2498f45ce267c3', 'request_id': 'req-4b907375-73cc-43ad-ae10-b80ab352002f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.925814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:35.927963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f245e72ca84936ae2498f45ce267c3', 'request_id': 'req-4b907375-73cc-43ad-ae10-b80ab352002f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.935314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f245e72ca84936ae2498f45ce267c3', 'request_id': 'req-4b907375-73cc-43ad-ae10-b80ab352002f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.947803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] The token cache is being invalidate because project 01f245e72ca84936ae2498f45ce267c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:35.948088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.948764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:35.950364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1129/2292] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:35 2022] DELETE /identity/v3/projects/01f245e72ca84936ae2498f45ce267c3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:35.954313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:35.964656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dd650-347e-4b9e-a3c2-d74aa8a8fddc service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:36.025688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:36.026389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.026665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.026940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] PATH_INFO: `/v3/users/8db4a7f0f6ea4f77935673856cb5f8f7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.028063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db4a7f0f6ea4f77935673856cb5f8f7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:36.030132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:36.094551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db4a7f0f6ea4f77935673856cb5f8f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.096746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:36.103343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db4a7f0f6ea4f77935673856cb5f8f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.122643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db4a7f0f6ea4f77935673856cb5f8f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.132078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db4a7f0f6ea4f77935673856cb5f8f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.133351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1a3fe246-670a-46a4-9db6-2e45d6e12f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db4a7f0f6ea4f77935673856cb5f8f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.142307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1164/2293] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:35 2022] DELETE /identity/v3/users/8db4a7f0f6ea4f77935673856cb5f8f7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:36.520881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:36.522407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:36.547897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b907375-73cc-43ad-ae10-b80ab352002f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:36.631722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:36.632432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.632773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.633154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] PATH_INFO: `/v3/projects/c52140ea5b1246d2939f12b55726cd01` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.634268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52140ea5b1246d2939f12b55726cd01)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:36.636238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:36.686862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52140ea5b1246d2939f12b55726cd01', 'request_id': 'req-876fa7d9-e5cb-42df-a993-849fc465f06b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.692047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52140ea5b1246d2939f12b55726cd01', 'request_id': 'req-876fa7d9-e5cb-42df-a993-849fc465f06b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.700943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52140ea5b1246d2939f12b55726cd01', 'request_id': 'req-876fa7d9-e5cb-42df-a993-849fc465f06b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.714097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] The token cache is being invalidate because project c52140ea5b1246d2939f12b55726cd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:36.714399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.715059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-876fa7d9-e5cb-42df-a993-849fc465f06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:36.716542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1130/2294] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:36 2022] DELETE /identity/v3/projects/c52140ea5b1246d2939f12b55726cd01 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:36.722247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.722536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.722824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.741897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:36.743078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:36.802990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1165/2295] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:36.810997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.811352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.811697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.831622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:36.832993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:36.841337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1131/2296] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:36.848840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:36.858970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8827f7-e141-4c34-a8d5-139108043f18 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:36.864565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:36.865243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.865525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.865796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.866537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:36.866843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:36.868586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:36.881607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1166/2297] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:53:36.886179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:36.896025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11a9b1-d832-4352-ae4d-97be110b56e1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:36.900686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:36.901284 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:36.901563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:36.901836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:36.902502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:36.904439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:36.932877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1132/2298] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:36 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:36.937002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:36.948818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c37c3-f505-4478-8be3-f0f2d83c76f6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.013828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.014432 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.014712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.014985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.015614 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.017597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.039901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1167/2299] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:36 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:37.048968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.061446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ffdc1-2ced-4b6c-a4a4-23443d25c6f4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.071146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.071780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.072065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.072353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.073056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.074890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.088528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1133/2300] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:37.090711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.101167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45de631-1750-43d2-b84e-16e1cd5e5ec8 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.105714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.106303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.106581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.106845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.107473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.109087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75add2-ec59-4d49-818d-0a470776b654 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.119850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1168/2301] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:37.123877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.133790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3739d-8fca-467f-8dec-b7b016a5c295 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.138918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.139565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.139845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.140527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] PATH_INFO: `/v3/projects/fe1deec7bbbf49e2bca650ac3bf8fbf5/users/716886b44bbe4293ace9d021d4608af5/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.165002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1deec7bbbf49e2bca650ac3bf8fbf5, user_id=716886b44bbe4293ace9d021d4608af5, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.168134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.182272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1134/2302] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:37 2022] PUT /identity/v3/projects/fe1deec7bbbf49e2bca650ac3bf8fbf5/users/716886b44bbe4293ace9d021d4608af5/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:37.192954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.193242 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.193512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.234378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] MFA Rules not processed for user `716886b44bbe4293ace9d021d4608af5`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:37.236171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:37.261519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:37.321277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1169/2303] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:37.332347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.335206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:37.346562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:37.403844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5036b-9f1c-4015-988f-7e86624e0953 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.408030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.408673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.408957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.409263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.418026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.419467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3371236-2229-4044-8ed6-edfbf00d29e0 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.428245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1135/2304] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:37.536820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.538514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:37.621122 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51166fa5-d0dc-4b70-b159-30443f2b1f48 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:37.626425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:37.627065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.627589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.627589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] PATH_INFO: `/v3/users/38c3e6abc0034f8d95bdcc9b8db71526` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.629036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c3e6abc0034f8d95bdcc9b8db71526)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:37.631446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:37.700056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c3e6abc0034f8d95bdcc9b8db71526'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:37.712702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c3e6abc0034f8d95bdcc9b8db71526'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:37.731302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c3e6abc0034f8d95bdcc9b8db71526'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:37.740072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c3e6abc0034f8d95bdcc9b8db71526'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:37.740980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c3e6abc0034f8d95bdcc9b8db71526'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:37.748524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1170/2305] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:37 2022] DELETE /identity/v3/users/38c3e6abc0034f8d95bdcc9b8db71526 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:37.753326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:37.754408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:37.755489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:37.782424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:37.785589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:37.903455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1136/2306] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:37.916461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:37.918932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:37.980389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:38.014521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe267dd-4bb1-465f-82fa-e84924e4f2e4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:38.018506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:38.019244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:38.019822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:38.020097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:38.029784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:38.031356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:38.043144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1171/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:38.305719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:38.307170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:38.319530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb519a2d-baba-454d-970e-c8c46f488125 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:38.324088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:38.324743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:38.325025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:38.325349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] PATH_INFO: `/v3/projects/d79626e4fcef49c784aeb5fdad057fe5` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:38.326400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d79626e4fcef49c784aeb5fdad057fe5)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:38.328300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:38.373819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79626e4fcef49c784aeb5fdad057fe5', 'request_id': 'req-5a990be7-2098-4f48-a333-07f56ae323e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.379253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79626e4fcef49c784aeb5fdad057fe5', 'request_id': 'req-5a990be7-2098-4f48-a333-07f56ae323e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.387534 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79626e4fcef49c784aeb5fdad057fe5', 'request_id': 'req-5a990be7-2098-4f48-a333-07f56ae323e8'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.399537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] The token cache is being invalidate because project d79626e4fcef49c784aeb5fdad057fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:38.399852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.400495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.402002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1137/2308] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:38 2022] DELETE /identity/v3/projects/d79626e4fcef49c784aeb5fdad057fe5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:38.406112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:38.415756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b18fb-b400-47fd-be99-cfd720cdba72 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:38.479272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:38.479908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:38.480187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:38.480482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] PATH_INFO: `/v3/users/162e9aed91ca49b388b7370d3a4ab99e` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:38.481698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162e9aed91ca49b388b7370d3a4ab99e)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:38.483695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:38.529921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162e9aed91ca49b388b7370d3a4ab99e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.537767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162e9aed91ca49b388b7370d3a4ab99e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.549864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162e9aed91ca49b388b7370d3a4ab99e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.557796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162e9aed91ca49b388b7370d3a4ab99e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.558397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162e9aed91ca49b388b7370d3a4ab99e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:38.564683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1172/2309] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:38 2022] DELETE /identity/v3/users/162e9aed91ca49b388b7370d3a4ab99e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:39.165169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:39.167667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:39.185365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a990be7-2098-4f48-a333-07f56ae323e8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:39.225948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:39.269280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:39.271246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:39.271526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:39.271796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] PATH_INFO: `/v3/projects/016775e86a454cf091353c8232df754c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:39.277256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016775e86a454cf091353c8232df754c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:39.279257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:39.354749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016775e86a454cf091353c8232df754c', 'request_id': 'req-41df7f61-38fc-4ab4-a4d8-45c0f2593914'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.362009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016775e86a454cf091353c8232df754c', 'request_id': 'req-41df7f61-38fc-4ab4-a4d8-45c0f2593914'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.370541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016775e86a454cf091353c8232df754c', 'request_id': 'req-41df7f61-38fc-4ab4-a4d8-45c0f2593914'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.383873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] The token cache is being invalidate because project 016775e86a454cf091353c8232df754c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:39.384173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.384896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.386542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1138/2310] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:39 2022] DELETE /identity/v3/projects/016775e86a454cf091353c8232df754c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:39.390732 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:39.392342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:39.407804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0518-7800-498b-b6d7-b7b7c7be1e49 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:39.466671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:39.467300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:39.467594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:39.467855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] PATH_INFO: `/v3/users/910a18fcd7534538b164e81eecdc04d7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:39.468996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910a18fcd7534538b164e81eecdc04d7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:39.470909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:39.473179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:39.518556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910a18fcd7534538b164e81eecdc04d7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.526323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910a18fcd7534538b164e81eecdc04d7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.537916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910a18fcd7534538b164e81eecdc04d7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.545063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910a18fcd7534538b164e81eecdc04d7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.545684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3056b60a-18a9-4cb0-a166-6c2a8b8a2de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910a18fcd7534538b164e81eecdc04d7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.551565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1173/2311] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:39 2022] DELETE /identity/v3/users/910a18fcd7534538b164e81eecdc04d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:39.821900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:39.823398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:39.837059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df7f61-38fc-4ab4-a4d8-45c0f2593914 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:39.892135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:39.892938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:39.893229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:39.893579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] PATH_INFO: `/v3/projects/c84b3ef5482d47499a263933eaabc4a1` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:39.895078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84b3ef5482d47499a263933eaabc4a1)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:39.897666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:39.942272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84b3ef5482d47499a263933eaabc4a1', 'request_id': 'req-35def219-a87f-429f-a835-98e8c25d62a3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.947522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84b3ef5482d47499a263933eaabc4a1', 'request_id': 'req-35def219-a87f-429f-a835-98e8c25d62a3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.955160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84b3ef5482d47499a263933eaabc4a1', 'request_id': 'req-35def219-a87f-429f-a835-98e8c25d62a3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.957150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:39.970547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] The token cache is being invalidate because project c84b3ef5482d47499a263933eaabc4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:39.970941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.971789 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-35def219-a87f-429f-a835-98e8c25d62a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:39.973891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1139/2312] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:39 2022] DELETE /identity/v3/projects/c84b3ef5482d47499a263933eaabc4a1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:39.978949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:39.979236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:39.979511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:39.998904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:40.000098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:40.059366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1174/2313] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:40.067194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.067487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.067767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.088859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:40.090082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:40.097911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1140/2314] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:40.102620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.112516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1c7e0-b834-4ff5-b7ce-9dd992e00923 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.117464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.117943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.118221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.118491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.119267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:40.119993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.121177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.133931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1175/2315] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:40.137725 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.148325 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabb9e6-1857-422f-abd7-f746ea86b649 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.157557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.158172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.158448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.158711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.159381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.161919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.189917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1141/2316] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:40 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:40.193977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.204063 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078ff72-247d-452d-a2b9-9063315b9afa admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.261733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.262344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.262627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.262901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.263541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.265478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.282811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:40.289505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1176/2317] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:40 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:40.293449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.303526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec4a91-45cc-4aab-95f9-a497a9dfa55c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.308070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.308679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.308966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.309234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.309865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.311443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.322023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1142/2318] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:40.325895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.336261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee7d3-2c29-45e4-8454-4b1ee8b4391b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.340964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.341553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.341839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.342109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.342740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.344333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25657810-df39-43b3-b7a7-83b27beaedbe admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.354967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1177/2319] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:40.359973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.370007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53d62-b8d6-4b24-ba28-a95dfde1de30 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.374616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.375318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.375615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.375891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] PATH_INFO: `/v3/projects/69ba32102f6340dea28abf8bf53a2b5d/users/72e8cf9141574729bb05bf70f0af3e84/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.390559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ba32102f6340dea28abf8bf53a2b5d, user_id=72e8cf9141574729bb05bf70f0af3e84, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.393270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.406597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1143/2320] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:40 2022] PUT /identity/v3/projects/69ba32102f6340dea28abf8bf53a2b5d/users/72e8cf9141574729bb05bf70f0af3e84/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:40.416840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.417317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.417606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.444930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] MFA Rules not processed for user `72e8cf9141574729bb05bf70f0af3e84`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:40.446177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:40.510087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1178/2321] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:40.521995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:40.523703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:40.578486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb4ea6-72a6-40f1-9257-ca1419cf7a78 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:40.582130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:40.582750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:40.583030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:40.583305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:40.591376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:40.592823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bf15b-e587-4aef-bd9b-1c1becef067b service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:40.601421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1144/2322] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:45.261904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:45.333008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cef035-4ba5-437e-8b50-1bc0d00f0137 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:45.347852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:45.348507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:45.348841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:45.349118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] PATH_INFO: `/v3/users/72e8cf9141574729bb05bf70f0af3e84` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:45.350362 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e8cf9141574729bb05bf70f0af3e84)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:45.352405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:45.407557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e8cf9141574729bb05bf70f0af3e84'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:45.415682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e8cf9141574729bb05bf70f0af3e84'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:45.438955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e8cf9141574729bb05bf70f0af3e84'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:45.443738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:45.448954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e8cf9141574729bb05bf70f0af3e84'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:45.450425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e8cf9141574729bb05bf70f0af3e84'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:45.461356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1179/2323] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:45 2022] DELETE /identity/v3/users/72e8cf9141574729bb05bf70f0af3e84 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:45.465651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:45.466555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:45.466979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:45.486930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:45.488409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:45.559137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1145/2324] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:45.570247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:45.571743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:45.650333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a925c-4c5a-4128-9c44-3a4547d21026 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:45.654456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:45.655119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:45.655700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:45.655700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:45.669958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:45.671634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460704a2-43e4-4a7b-bbb4-31017b689902 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:45.685085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1180/2325] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:45.997253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:45.998677 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:46.022408 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2638f-4b01-4f95-96c4-e792add48bd1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.030949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.032221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.032880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.033541 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] PATH_INFO: `/v3/projects/69ba32102f6340dea28abf8bf53a2b5d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.035290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ba32102f6340dea28abf8bf53a2b5d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.038409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.090400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:46.113128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ba32102f6340dea28abf8bf53a2b5d', 'request_id': 'req-a18be8f0-28fe-49b1-9247-c1f0766b4f17'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:46.122062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ba32102f6340dea28abf8bf53a2b5d', 'request_id': 'req-a18be8f0-28fe-49b1-9247-c1f0766b4f17'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:46.136177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ba32102f6340dea28abf8bf53a2b5d', 'request_id': 'req-a18be8f0-28fe-49b1-9247-c1f0766b4f17'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:46.151518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] The token cache is being invalidate because project 69ba32102f6340dea28abf8bf53a2b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:46.152038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:46.153244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a18be8f0-28fe-49b1-9247-c1f0766b4f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:46.155042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1146/2326] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:45 2022] DELETE /identity/v3/projects/69ba32102f6340dea28abf8bf53a2b5d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:46.159873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.160167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.160458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.185036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:46.186241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:46.253188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1181/2327] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:46.262214 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.262591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.262932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.288295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:46.289770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:46.301421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1147/2328] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:46.302684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.314182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd031ee-4fad-445a-8986-43abf3ca2986 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.319660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.320391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.320893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.321238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.322003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:46.322305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.323884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.339241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1182/2329] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:46.340385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.350352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ae218-cd39-4b71-811f-d44fef68fe1f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.356703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.357431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.357769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.357979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.358622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.360862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.393750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1148/2330] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:46 2022] POST /identity/v3/projects => generated 397 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:46.397385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.411757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84e44a-adc4-4b57-aaaa-477207cf17d1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.481484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.482093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.482376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.482777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.483397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.485363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.507409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1183/2331] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:46 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:46.512508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.523997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac8fd8-9fef-42ad-bc70-3deeca707950 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.528933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.529542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.529998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.530410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.531263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.533639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.557394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1149/2332] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:46.562841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.573795 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5928c8-0fc6-4e11-8233-c0f129b637cc admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.582131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.582992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.583498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.584810 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.585659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.588261 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfb10e-e6b0-44df-b2e4-d2092dffc549 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.603731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1184/2333] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:46.608353 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.620202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758694f-f5d7-46b0-9ff9-2703b01490f7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.641201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.641201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.641201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.641201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] PATH_INFO: `/v3/projects/7bd80fc4af4b4b9c9055fae13e6f9506/users/b56357c075d64d97a818c7ff47f3e76b/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.674245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd80fc4af4b4b9c9055fae13e6f9506, user_id=b56357c075d64d97a818c7ff47f3e76b, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.678815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.695226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1150/2334] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:46 2022] PUT /identity/v3/projects/7bd80fc4af4b4b9c9055fae13e6f9506/users/b56357c075d64d97a818c7ff47f3e76b/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:46.707631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.708399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.710300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.719215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:46.752569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] MFA Rules not processed for user `b56357c075d64d97a818c7ff47f3e76b`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:46.753947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:46.827649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1185/2335] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:46 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:46.839517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:46.841368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:46.917010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00037f63-3479-4e27-b7bc-2bb37074e26a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:46.921598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:46.922224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:46.922502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:46.922771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:46.932486 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:46.934007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5ba9f-5e0c-4b19-b3f7-c0fe1f759fc6 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:46.944488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1151/2336] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:52.144861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:52.146827 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:52.213603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141ec62-7525-467c-acd8-737f3d686110 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:52.218039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:52.218660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:52.218935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:52.219425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] PATH_INFO: `/v3/users/ba20dd690f504b6d96eda957f634d763` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:52.220314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba20dd690f504b6d96eda957f634d763)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:52.222290 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:52.276040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba20dd690f504b6d96eda957f634d763'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.286850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba20dd690f504b6d96eda957f634d763'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.300368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba20dd690f504b6d96eda957f634d763'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.309364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba20dd690f504b6d96eda957f634d763'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.309982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba20dd690f504b6d96eda957f634d763'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.316919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1186/2337] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:52 2022] DELETE /identity/v3/users/ba20dd690f504b6d96eda957f634d763 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:52.322168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:52.322655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:52.322870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:52.342662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:52.344074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:52.397394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:52.412727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1152/2338] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:52.418238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:52.419686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:52.434364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:52.483150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05895aa-04f4-492f-a3dd-8eb0b79892e6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:52.486967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:52.487580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:52.487842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:52.488114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:52.502144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:52.504326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5723060-594f-4320-a8a2-494b63a83747 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:52.516665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1187/2339] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:52.903484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:52.904876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:52.918467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbdeee-a551-41cd-9465-67e049dfd837 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:52.924818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:52.925688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:52.926040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:52.926388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] PATH_INFO: `/v3/projects/daec5c8157244e228263e82116888f96` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:52.927764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daec5c8157244e228263e82116888f96)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:52.929780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:52.982143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daec5c8157244e228263e82116888f96', 'request_id': 'req-6c127986-ddc3-40ee-990a-06211d646789'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.987929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daec5c8157244e228263e82116888f96', 'request_id': 'req-6c127986-ddc3-40ee-990a-06211d646789'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:52.995441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daec5c8157244e228263e82116888f96', 'request_id': 'req-6c127986-ddc3-40ee-990a-06211d646789'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.007259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] The token cache is being invalidate because project daec5c8157244e228263e82116888f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:53.007548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.008190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-6c127986-ddc3-40ee-990a-06211d646789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.009705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1153/2340] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:52 2022] DELETE /identity/v3/projects/daec5c8157244e228263e82116888f96 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:53.014701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.015036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.015390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.036301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:53.037654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.102574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1188/2341] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:53.111283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.111572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.111850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.135068 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:53.137039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.151743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1154/2342] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:53.161753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.172132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aac77-2fa1-495d-a6e1-460852e21100 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.178291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.178291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.178731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.178833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.179406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:53.179714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.182351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.197462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1189/2343] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:53.201909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.212260 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ff2eb-6f47-4fa7-aef7-ef2862b34fd7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.217304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.217942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.218191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.218462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.219279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.221361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.251911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1155/2344] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:53.256852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.266821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a151720-f59f-43b5-9d36-a392719b6d6c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.324124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.324758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.325040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.325314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.325948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.327882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.351908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1190/2345] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:53.353894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.367375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc56e0-6147-481d-b71c-04b74b7cc5cd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.373632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.374702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.375150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.375299 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.376138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.378242 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.393900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1156/2346] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:53.395835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.430478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450391af-8caa-4602-a9eb-acf7fd0bd3ba admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.435287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.435895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.436175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.436452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.437125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.438853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbf338-f258-4042-ba71-00f1b80cc3fd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.450374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1191/2347] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:53.454766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.467264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230524-5cf5-4476-bb81-0cdcadd31332 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.472787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.473457 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.473741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.474016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] PATH_INFO: `/v3/projects/44afe4956dec4d7f9ba3e3bcb13a8f71/users/5e65bab508d14d2aa026ee71230c8f8d/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.489212 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=44afe4956dec4d7f9ba3e3bcb13a8f71, user_id=5e65bab508d14d2aa026ee71230c8f8d, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.493377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.507919 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1157/2348] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:53 2022] PUT /identity/v3/projects/44afe4956dec4d7f9ba3e3bcb13a8f71/users/5e65bab508d14d2aa026ee71230c8f8d/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:53.521161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.521490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.521770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.550285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] MFA Rules not processed for user `5e65bab508d14d2aa026ee71230c8f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:53.551497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.561622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:53.619934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1192/2349] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:53.624487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.690202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fbd293-193a-4d9c-9e6f-6328f7f3e95a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.694891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.695492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.695772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.696044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.696720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.698629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2fd96-b8e8-4c9c-b0eb-a60fff979f05 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.735551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1158/2350] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:53.740876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.752351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f7340-97bb-4e68-a91b-b4df73895497 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.764943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da533-6b0f-4577-af2a-7ab62f3fa358 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.764943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da533-6b0f-4577-af2a-7ab62f3fa358 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.764943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14da533-6b0f-4577-af2a-7ab62f3fa358 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.766575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1159/2351] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 20 04:53:53.798072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.798861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.799811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.810074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:53.814461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.815223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.815509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.815780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.816426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.818506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.828323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:53.830499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.846705 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1193/2352] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:53.848292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.851463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.863262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8790a-f1ed-441f-8228-6bd1aca43203 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.868696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.869319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.869597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.869865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] PATH_INFO: `/v3/users/e1ba955e328a40318a36056695c6fc03` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.870987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ba955e328a40318a36056695c6fc03)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.872946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.898753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1160/2353] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:53:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:53.899645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.922494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e47411-bbbc-48e0-a29e-39a5de1386f3 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.931072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.931762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.932108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.932443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.934190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.935882 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:53.939343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ba955e328a40318a36056695c6fc03'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.952273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1161/2354] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:53.952273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.952273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.955582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ba955e328a40318a36056695c6fc03'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.970579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ba955e328a40318a36056695c6fc03'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.970752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137213d9-65d3-4878-ac56-a8c082b851ad admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:53.976717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:53.977422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:53.977706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:53.977979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:53.979212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ba955e328a40318a36056695c6fc03'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.980061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ba955e328a40318a36056695c6fc03'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:53.986571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1194/2355] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:53 2022] DELETE /identity/v3/users/e1ba955e328a40318a36056695c6fc03 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:53.987177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:53.989602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:53.990680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:53.991918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.002358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1162/2356] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:54.008885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:54.067447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ce26f-eb06-4f3e-8acc-d53ff7185bec admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.073374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.074125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.075148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.075148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.075453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.077681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba9e4f-a685-45b6-a963-6fa6b6fe7ada admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.088020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1195/2357] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:54.093093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.107884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da18961-5e07-46ab-8343-e66cceecf9b4 service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.122450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.123135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.123909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.124579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] PATH_INFO: `/v3/projects/af122bd30c3b42d9a0fa12d4876a211d/users/65cd8fce71854c098023bd0a50c5954c/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.143610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af122bd30c3b42d9a0fa12d4876a211d, user_id=65cd8fce71854c098023bd0a50c5954c, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.147407 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.172641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1163/2358] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:54 2022] PUT /identity/v3/projects/af122bd30c3b42d9a0fa12d4876a211d/users/65cd8fce71854c098023bd0a50c5954c/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:54.179236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.179709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.179806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.214986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] MFA Rules not processed for user `65cd8fce71854c098023bd0a50c5954c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:54.216331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.220027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.222732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.320180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb6e65-ea30-4410-b4c8-bdae3189077d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.323548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1196/2359] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:54.327034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.327535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.327932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.328134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] PATH_INFO: `/v3/projects/b2cac3cfcbea45d1b253b1c5d566bed9` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.328247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.329366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2cac3cfcbea45d1b253b1c5d566bed9)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.331731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.338869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9537-1a5b-4a4b-a039-6dca008453b8 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.343540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.344174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.344457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.344768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.345400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.347297 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.377015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1197/2360] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:54 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:54.381526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.391807 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2cac3cfcbea45d1b253b1c5d566bed9', 'request_id': 'req-c4195608-953b-4021-8b8c-f3afd8719fac'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.394261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:54.399208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2cac3cfcbea45d1b253b1c5d566bed9', 'request_id': 'req-c4195608-953b-4021-8b8c-f3afd8719fac'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.407580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2cac3cfcbea45d1b253b1c5d566bed9', 'request_id': 'req-c4195608-953b-4021-8b8c-f3afd8719fac'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.422168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] The token cache is being invalidate because project b2cac3cfcbea45d1b253b1c5d566bed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:54.422464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.423574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.425387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1164/2361] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:54 2022] DELETE /identity/v3/projects/b2cac3cfcbea45d1b253b1c5d566bed9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:54.429860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.431590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.454571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baa23b-af1d-4add-9297-496f929201c2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.460730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.461428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.461789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.462173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.463504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.465626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.496167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1198/2362] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:54 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:54.497622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.500350 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.514658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc98-31fc-4d33-8cb9-161137a60411 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.515186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4195608-953b-4021-8b8c-f3afd8719fac admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.521251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.521606 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.522056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.522223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.522459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.522571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.522746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.522881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] PATH_INFO: `/v3/users/5c8115ec2055401094f5a06f3e89672f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.523448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.524123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8115ec2055401094f5a06f3e89672f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.525017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.526238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.526922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:54.540820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1199/2363] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:54.544966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.562279 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b62ff-4a8f-4197-a8e0-44121e18d2a5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.569973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.571083 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.571391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.572682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] PATH_INFO: `/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f/users/7874dfdac61540b9a03a5cb1c6b65b0d/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.585231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8115ec2055401094f5a06f3e89672f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.590839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7aeb95cfb54bed9e608d58322b6e9f, user_id=7874dfdac61540b9a03a5cb1c6b65b0d, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.592818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8115ec2055401094f5a06f3e89672f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.593664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.607475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8115ec2055401094f5a06f3e89672f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.613442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1200/2364] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:54 2022] PUT /identity/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f/users/7874dfdac61540b9a03a5cb1c6b65b0d/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:54.617103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8115ec2055401094f5a06f3e89672f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.617103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8115ec2055401094f5a06f3e89672f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.618161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.619942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.622477 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1165/2365] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:54 2022] DELETE /identity/v3/users/5c8115ec2055401094f5a06f3e89672f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:54.687969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7502a0f-7c7e-4920-be93-57938c91aa23 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.749153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.749775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.750055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.750371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.751018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.752848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.764455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1201/2366] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:54.769278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.781924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880e1f-3fcd-462f-bf9f-0e2fab5644bf admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.792879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.793607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.793909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.794276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] PATH_INFO: `/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f/users/7874dfdac61540b9a03a5cb1c6b65b0d/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.798340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7aeb95cfb54bed9e608d58322b6e9f, user_id=7874dfdac61540b9a03a5cb1c6b65b0d, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.804182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbca388-1f5b-41b9-a7a9-76dffa7cea2f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.805443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.807652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.819841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1166/2367] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:54 2022] PUT /identity/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f/users/7874dfdac61540b9a03a5cb1c6b65b0d/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:54.831023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.831366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.831643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.833255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d408a-0d62-4669-8980-303d6090dace admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:54.861301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] MFA Rules not processed for user `7874dfdac61540b9a03a5cb1c6b65b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:54.862518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.897435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:54.899511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:54.900159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:54.900441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:54.900747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] PATH_INFO: `/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:54.901884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852321a8a8e84ca1ab2d9fd7bff868a2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:54.903824 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:54.929917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:54.943817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1167/2368] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:54.956750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:54.959543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:54.971856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852321a8a8e84ca1ab2d9fd7bff868a2', 'request_id': 'req-bb1e2a5d-429d-4c43-b891-a2d61a418761'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.981213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852321a8a8e84ca1ab2d9fd7bff868a2', 'request_id': 'req-bb1e2a5d-429d-4c43-b891-a2d61a418761'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:54.988530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852321a8a8e84ca1ab2d9fd7bff868a2', 'request_id': 'req-bb1e2a5d-429d-4c43-b891-a2d61a418761'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.001231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] The token cache is being invalidate because project 852321a8a8e84ca1ab2d9fd7bff868a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:55.001521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.002154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-bb1e2a5d-429d-4c43-b891-a2d61a418761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.003647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1202/2369] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:54 2022] DELETE /identity/v3/projects/852321a8a8e84ca1ab2d9fd7bff868a2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:55.008494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de078b-1896-4348-b09b-112f1e6c13d0 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.008805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de078b-1896-4348-b09b-112f1e6c13d0 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.009080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de078b-1896-4348-b09b-112f1e6c13d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.023733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd8a72-6204-4786-8bd4-17289175e8ff None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.029994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-06de078b-1896-4348-b09b-112f1e6c13d0 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:55.031197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-06de078b-1896-4348-b09b-112f1e6c13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.072500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.073140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.073419 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.073691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.075433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.094141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1203/2370] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:55.102311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.102597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.102870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.126326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:55.127542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.138601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1204/2371] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:55.139538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.142345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.143992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.149180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09022f00-d263-4380-b13f-979a58911dc5 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.153840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.154437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.154714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.154985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.155695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1168/2372] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:55.155899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:55.156222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.157920 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.170968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1205/2373] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:53:55.175312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.186272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b6e4b-c5bb-4888-b903-ad4e35dfe92a service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.191083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.191616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.191985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.192163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.193349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.195266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.225087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1169/2374] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:55.229647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.241172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c04a72-4368-46c3-8f1b-25ecc86da0f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.309091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.309707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.309984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.310265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.310892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.313036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.338051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1206/2375] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:53:55.342627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.359441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452adc7-2078-4e28-bfa2-e618a490ea19 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.377879 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.379314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.379759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.380560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.381485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.383479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.395637 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1170/2376] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:55.399957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.411313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409691-d35f-465d-9d81-0e43154ca8a6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.417154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.417885 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.418234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.428825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.429542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.431172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7dcff-cc34-4405-8ab8-2d260c6da7c6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.447511 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1207/2377] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:55.451941 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.462678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38a3c7-a31c-4472-b7c2-244b163a8383 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.467931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.468620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.469023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.469274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] PATH_INFO: `/v3/projects/0e251ef14bba4f84bf4a8fda3d10d431/users/76275813aa5448028fdef844f96abb69/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.474721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:55.490826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e251ef14bba4f84bf4a8fda3d10d431, user_id=76275813aa5448028fdef844f96abb69, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.493665 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.511070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1171/2378] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:55 2022] PUT /identity/v3/projects/0e251ef14bba4f84bf4a8fda3d10d431/users/76275813aa5448028fdef844f96abb69/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:55.523710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.523995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.524266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.532305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:55.559265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] MFA Rules not processed for user `76275813aa5448028fdef844f96abb69`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:55.560619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.625389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1208/2379] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:55.638737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.640590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.700338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d775977-28e3-467f-adfe-0fde0a9b0c57 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.704000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.704618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.704903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.705171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.713805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.715250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e84e7a-266c-4b29-bc51-418b4bd6debf service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.724239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1172/2380] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:55.746030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:55.811353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ab81a-b1f1-491d-9515-5378a092393b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:55.816073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:55.816721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.817029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.817319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] PATH_INFO: `/v3/users/76275813aa5448028fdef844f96abb69` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.818978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] RBAC: Authorizing `identity:delete_user(user_id=76275813aa5448028fdef844f96abb69)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:55.820937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:55.868830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76275813aa5448028fdef844f96abb69'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.876191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76275813aa5448028fdef844f96abb69'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.889697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76275813aa5448028fdef844f96abb69'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.898288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76275813aa5448028fdef844f96abb69'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.898947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76275813aa5448028fdef844f96abb69'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:55.906046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1209/2381] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:53:55 2022] DELETE /identity/v3/users/76275813aa5448028fdef844f96abb69 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:55.910127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:55.910468 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:55.910744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:55.932802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:55.934058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:55.998756 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1173/2382] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:56.008743 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.010208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.027319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:56.089189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0181ea0-f961-4430-ac87-81100d5b547c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.092832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.093470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.093873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.094149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.104419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:56.105888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912d1d6-9849-45d5-b276-c04a27626875 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:56.116408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1210/2383] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:56.418113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.419546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.440803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ead88-8587-4f18-9fc0-ff969f184a17 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.449481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.450315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.450770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.451490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] PATH_INFO: `/v3/projects/0e251ef14bba4f84bf4a8fda3d10d431` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.452823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e251ef14bba4f84bf4a8fda3d10d431)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:56.455570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:56.508817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e251ef14bba4f84bf4a8fda3d10d431', 'request_id': 'req-3ca8f75b-8ca8-418b-8cbb-acf32369a202'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:56.518833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e251ef14bba4f84bf4a8fda3d10d431', 'request_id': 'req-3ca8f75b-8ca8-418b-8cbb-acf32369a202'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:56.529223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e251ef14bba4f84bf4a8fda3d10d431', 'request_id': 'req-3ca8f75b-8ca8-418b-8cbb-acf32369a202'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:56.542982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] The token cache is being invalidate because project 0e251ef14bba4f84bf4a8fda3d10d431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:53:56.543321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:56.543976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3ca8f75b-8ca8-418b-8cbb-acf32369a202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:53:56.545892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1174/2384] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:53:56 2022] DELETE /identity/v3/projects/0e251ef14bba4f84bf4a8fda3d10d431 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:53:56.550959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.551408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.551489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.572946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:56.574212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.656742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1211/2385] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:53:56.664564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.665184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.665465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.693240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:56.694883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.721210 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1175/2386] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:56.736932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.750061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17a423-1e69-4def-a166-9da1d68a90b1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.756372 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.757149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.757442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.757726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.758430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:53:56.759938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:56.762175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:56.784542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1212/2387] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:53:56.785948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.786442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.788353 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.792224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:56.797890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7d63b-af26-4171-8af9-c63b818fc32e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.823350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.826393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.826948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.827235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.830575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:56.836052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:56.862780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa27a8-47a6-47aa-aa19-a8fd8dc04d93 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.882734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.883856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.884282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.884730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.887279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1176/2388] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:53:56 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:56.887878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:56.891255 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:56.989237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c779e-a96a-444d-8b2e-c5806ea0d1ec admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:56.994661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:56.995703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:56.996154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:56.996586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:56.997456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.000492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.007803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.009807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.013951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:57.025903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1213/2389] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:53:57.034217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1177/2390] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:53:56 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:53:57.038794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:57.049431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99935ce8-3b67-4a59-9dee-630f76f75977 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:57.055041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:57.055637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:57.055922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:57.056194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:57.056856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.058455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.069776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1214/2391] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:57.073917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:57.084087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe979a26-b15c-4ee0-882f-5c07a8a4748e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:57.089032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:57.089630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:57.089935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:57.090204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:57.090841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.092417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a44ef2-f817-4ed8-9c05-0a107d6be481 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.103236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1178/2392] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:53:57.107488 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:57.118500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a29da-d850-4187-8379-97f6dde329b0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:57.123413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:57.124062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:57.124369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:57.124694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] PATH_INFO: `/v3/projects/6076d331e7bc4a8da0255af513578ece/users/98ca2bd265d5456786cecc6d37a427df/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:57.140844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6076d331e7bc4a8da0255af513578ece, user_id=98ca2bd265d5456786cecc6d37a427df, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.143548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.161997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1215/2393] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:53:57 2022] PUT /identity/v3/projects/6076d331e7bc4a8da0255af513578ece/users/98ca2bd265d5456786cecc6d37a427df/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:53:57.172324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045face2-2083-4c7e-a834-4106280acaee None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:57.172628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045face2-2083-4c7e-a834-4106280acaee None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:57.173026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045face2-2083-4c7e-a834-4106280acaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:57.206531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-045face2-2083-4c7e-a834-4106280acaee None None] MFA Rules not processed for user `98ca2bd265d5456786cecc6d37a427df`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:53:57.207718 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-045face2-2083-4c7e-a834-4106280acaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:57.283002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1179/2394] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:53:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:53:57.354964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:57.356708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:57.413371 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2929cc2-6add-40bb-839e-6dbf849c0f51 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:57.416982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:57.417598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:57.417876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:57.418147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:57.426287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:57.428386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:57.436955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1216/2395] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:53:59.785859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045face2-2083-4c7e-a834-4106280acaee None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:53:59.796120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045face2-2083-4c7e-a834-4106280acaee None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:53:59.800948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:53:59.801695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:53:59.802832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:53:59.803142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:53:59.805089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:53:59.853194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:53:59.870991 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:53:59.872416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61485de-9888-4049-b9de-5cf9e85e2df5 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:53:59.882045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1180/2396] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:53:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:00.920142 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:00.921825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:00.998033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f888fa-ae30-4e64-83f9-08c01ac5c9ae service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.004816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.006199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.006540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.006867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] PATH_INFO: `/v3/users/716886b44bbe4293ace9d021d4608af5` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.008229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=716886b44bbe4293ace9d021d4608af5)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.010328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.060836 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '716886b44bbe4293ace9d021d4608af5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.068654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '716886b44bbe4293ace9d021d4608af5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.080090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:01.087638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '716886b44bbe4293ace9d021d4608af5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.097644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '716886b44bbe4293ace9d021d4608af5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.098460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '716886b44bbe4293ace9d021d4608af5'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.106093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1217/2397] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:00 2022] DELETE /identity/v3/users/716886b44bbe4293ace9d021d4608af5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:01.110916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.111443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.111443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.132964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:01.132964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.198922 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1181/2398] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:01.211071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.212497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.273569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442702e3-f7a7-4b5f-b19d-7fe23e0b7405 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.278460 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.279280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.279598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.279988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.290873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.292315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.302663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1218/2399] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:01.557399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.558875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.573227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03557a1-f755-47f3-a924-d274d1d315ea None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.578348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.579119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.579467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.579750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] PATH_INFO: `/v3/users/b56357c075d64d97a818c7ff47f3e76b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.581012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56357c075d64d97a818c7ff47f3e76b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.583248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.606505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.609010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.627567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35091cd-b5f3-4954-90c1-33cec0d33ba9 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.633179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.634982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.635274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.635580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] PATH_INFO: `/v3/projects/fe1deec7bbbf49e2bca650ac3bf8fbf5` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.637085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1deec7bbbf49e2bca650ac3bf8fbf5)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.638982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.669011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56357c075d64d97a818c7ff47f3e76b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.681487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56357c075d64d97a818c7ff47f3e76b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.696816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56357c075d64d97a818c7ff47f3e76b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.704651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1deec7bbbf49e2bca650ac3bf8fbf5', 'request_id': 'req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.707776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56357c075d64d97a818c7ff47f3e76b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.709297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-46214183-f0bd-49e7-b0de-734c7946287a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56357c075d64d97a818c7ff47f3e76b'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.712949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1deec7bbbf49e2bca650ac3bf8fbf5', 'request_id': 'req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.720285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1182/2400] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:01 2022] DELETE /identity/v3/users/b56357c075d64d97a818c7ff47f3e76b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:01.720779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1deec7bbbf49e2bca650ac3bf8fbf5', 'request_id': 'req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.726304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.727462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.727989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.742629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] The token cache is being invalidate because project fe1deec7bbbf49e2bca650ac3bf8fbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:01.743370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.744359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-48cf9a0b-6264-4c81-870e-934d5e7a2bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:01.746266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1219/2401] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:01 2022] DELETE /identity/v3/projects/fe1deec7bbbf49e2bca650ac3bf8fbf5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:01.750652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:01.751258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda8a1-34db-41ab-8031-efc8c060fb81 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.751864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.752004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda8a1-34db-41ab-8031-efc8c060fb81 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.752253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abda8a1-34db-41ab-8031-efc8c060fb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.773337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0abda8a1-34db-41ab-8031-efc8c060fb81 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:01.774568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0abda8a1-34db-41ab-8031-efc8c060fb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.815953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:01.832592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1183/2402] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:01.844381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.846107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.848624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1220/2403] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:01.856621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.857094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.858304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.886688 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:01.887921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:01.897608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1221/2404] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:01.901728 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.907515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:01.910999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69a92d-f0bf-4b2b-b0b3-9488ca676a69 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.915283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a815a-f015-44a8-82de-8d22715545a6 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.918486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.919413 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.919768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.920100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.920213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.920739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.920998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:54:01.921430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.921430 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.921613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.923401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.930618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.932062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.935518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1222/2405] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:54:01.939695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:01.945330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1184/2406] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:54:01.950212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5fa5d-a49f-41f9-bcdd-27a4693b9490 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:01.954847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:01.955440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:01.955722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:01.955996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:01.956646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:01.959223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:01.991453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1223/2407] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:54:01.992645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.002852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd3506-b473-431d-b259-88da73d69263 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.065352 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.065944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.066226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.066494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.067119 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.069607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.092065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1185/2408] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:01 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:02.096439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.107602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f01db8-d6c1-4c64-9018-e0d057ee722f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.112300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.112930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.113222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.113501 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.114138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.116401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.126596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1224/2409] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:02.130953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.142146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c82dd9-eafa-49b2-8dad-f051cc096ddd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.149013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.149624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.150022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.150308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.150963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.152568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541bda0-cbb0-4efd-9c29-274be8631df0 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.170897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1186/2410] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:02.171019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.182988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f660e-9040-480c-8704-02bd6fa7f309 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.187759 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.188410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.188736 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.189011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] PATH_INFO: `/v3/projects/e20f1eadf47949a5a51451bb06de2d41/users/2653e7bc5e174152a7c18b3635d9b549/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.205274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] RBAC: Authorizing `identity:create_grant(project_id=e20f1eadf47949a5a51451bb06de2d41, user_id=2653e7bc5e174152a7c18b3635d9b549, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.208266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.221249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1225/2411] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:02 2022] PUT /identity/v3/projects/e20f1eadf47949a5a51451bb06de2d41/users/2653e7bc5e174152a7c18b3635d9b549/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:02.232715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.233096 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.233378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.236646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.238569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.268763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] MFA Rules not processed for user `2653e7bc5e174152a7c18b3635d9b549`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:02.270076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.306698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d54773-49c5-4d17-9d33-844bc3d2aead admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.311451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.312108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.312671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.312671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] PATH_INFO: `/v3/projects/7bd80fc4af4b4b9c9055fae13e6f9506` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.313796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd80fc4af4b4b9c9055fae13e6f9506)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.315708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.336109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1187/2412] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:02.348195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.349649 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.360190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:02.362902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd80fc4af4b4b9c9055fae13e6f9506', 'request_id': 'req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:02.369278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd80fc4af4b4b9c9055fae13e6f9506', 'request_id': 'req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:02.375720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd80fc4af4b4b9c9055fae13e6f9506', 'request_id': 'req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:02.389170 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] The token cache is being invalidate because project 7bd80fc4af4b4b9c9055fae13e6f9506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:02.389451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:02.390138 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9ef8928a-10d0-43bb-a769-6b083dd0e6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:02.391734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1226/2413] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:02 2022] DELETE /identity/v3/projects/7bd80fc4af4b4b9c9055fae13e6f9506 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:54:02.397912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8d8f5-6ea1-487d-8183-0e5df22b3de4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.398263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8d8f5-6ea1-487d-8183-0e5df22b3de4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.398546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8d8f5-6ea1-487d-8183-0e5df22b3de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.410566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c304aa-d512-4f7c-82bc-4db317f7e694 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.417805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f8d8d8f5-6ea1-487d-8183-0e5df22b3de4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:02.419004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f8d8d8f5-6ea1-487d-8183-0e5df22b3de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.438053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:02.466564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.467360 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.467464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.467686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.469855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.502532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1227/2414] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:02.511848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.512207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.512492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.536167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.537639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.539904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:02.541325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.546839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1188/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:54:02.550171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1228/2416] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:02.554928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.565453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ed0bb-adaf-42a4-8e1e-940878dfbcf5 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.570216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.570808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.571092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.571367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.572002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:54:02.572305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.573947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.585826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1189/2417] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 20 04:54:02.590164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.599999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518d1ac-0084-4c0c-a74e-e2f7af1d2634 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.604592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.605228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.605508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.605777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.606396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.608283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.636434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1229/2418] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:54:02.640581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.651505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f400-e45d-47a4-9e8d-6afd4ba3ccda admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.712621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.713254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.713537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.713990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.714625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.717169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.729179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:02.746431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1190/2419] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:54:02.747473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.761172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef837fc-4ab4-4a76-8094-f8b7f530830f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.767255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.767856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.768133 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.768401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.769166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.771887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.789439 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.795482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1230/2420] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:54:02.801736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4edc1-f9d4-46a5-b9d5-295c46744d69 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.806980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.807586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.807868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.808177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.808971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.810604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85af534-5e88-4e38-9afa-927409ae0bf9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.823128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1191/2421] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:02.827090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.839756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68de465-0690-4641-80a7-a68e836c0415 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:02.844897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:02.845794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.846466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.846815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] PATH_INFO: `/v3/projects/2e65f62fae68435ea0304bebc2a21b90/users/4f45aecdffb641d696b8c05d676f71aa/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.871224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e65f62fae68435ea0304bebc2a21b90, user_id=4f45aecdffb641d696b8c05d676f71aa, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:02.874119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:02.887629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1231/2422] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:02 2022] PUT /identity/v3/projects/2e65f62fae68435ea0304bebc2a21b90/users/4f45aecdffb641d696b8c05d676f71aa/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:02.892765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:02.893054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:02.893389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:02.914435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:02.916680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:02.978844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1192/2423] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:02.990709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:02.993245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:03.052229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c1431-edf9-41c8-b3b8-1c31c25a1bc2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:03.056409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:03.057117 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:03.057515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:03.057862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:03.067357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:03.068931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad525d4-5add-4a62-9a06-f8e6a7032806 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:03.078594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1232/2424] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:06.384587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:06.386286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:06.399381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6d4f-64ad-467e-94ec-bae82b579dfb None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:06.404158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:06.404845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:06.405132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:06.405405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] PATH_INFO: `/v3/users/5e65bab508d14d2aa026ee71230c8f8d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:06.406586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e65bab508d14d2aa026ee71230c8f8d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:06.408583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:06.464985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e65bab508d14d2aa026ee71230c8f8d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:06.472546 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e65bab508d14d2aa026ee71230c8f8d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:06.490082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e65bab508d14d2aa026ee71230c8f8d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:06.498489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e65bab508d14d2aa026ee71230c8f8d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:06.499413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e65bab508d14d2aa026ee71230c8f8d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:06.506421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1193/2425] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:06 2022] DELETE /identity/v3/users/5e65bab508d14d2aa026ee71230c8f8d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:54:06.511205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:06.511706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:06.511843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:06.542755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:06.544044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:06.631782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1233/2426] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:06.657118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:06.659538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:06.737559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a64d97-504a-4d8e-8963-8a65b58ec54e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:06.750799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:06.751746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:06.752306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:06.752866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:06.764835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:06.766566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:06.774783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:06.781093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1194/2427] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:07.101221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:07.102686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:07.111418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:07.117518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fb0a8-18b1-4c6e-91f2-aa1dad1be3c4 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:07.123888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:07.124530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:07.124845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:07.125118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] PATH_INFO: `/v3/projects/44afe4956dec4d7f9ba3e3bcb13a8f71` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:07.126546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44afe4956dec4d7f9ba3e3bcb13a8f71)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:07.129338 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:07.200150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44afe4956dec4d7f9ba3e3bcb13a8f71', 'request_id': 'req-1027a63b-76ae-4219-826a-d9052e54ef02'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.207416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44afe4956dec4d7f9ba3e3bcb13a8f71', 'request_id': 'req-1027a63b-76ae-4219-826a-d9052e54ef02'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.215431 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44afe4956dec4d7f9ba3e3bcb13a8f71', 'request_id': 'req-1027a63b-76ae-4219-826a-d9052e54ef02'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.239913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] The token cache is being invalidate because project 44afe4956dec4d7f9ba3e3bcb13a8f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:07.240283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.241222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.242823 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1234/2428] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:07 2022] DELETE /identity/v3/projects/44afe4956dec4d7f9ba3e3bcb13a8f71 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:07.248184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:07.263043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8378a-a8ff-4127-ab4e-15cda6ac793c service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:07.381598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:07.382090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:07.382476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:07.382875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] PATH_INFO: `/v3/users/65cd8fce71854c098023bd0a50c5954c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:07.384409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65cd8fce71854c098023bd0a50c5954c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:07.386942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:07.490481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65cd8fce71854c098023bd0a50c5954c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.501415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65cd8fce71854c098023bd0a50c5954c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.524578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65cd8fce71854c098023bd0a50c5954c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.546173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65cd8fce71854c098023bd0a50c5954c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.546980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65cd8fce71854c098023bd0a50c5954c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:07.560698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1195/2429] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:07 2022] DELETE /identity/v3/users/65cd8fce71854c098023bd0a50c5954c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:08.005642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:08.007081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:08.028125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027a63b-76ae-4219-826a-d9052e54ef02 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:08.129364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:08.130904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:08.132107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:08.132401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] PATH_INFO: `/v3/projects/af122bd30c3b42d9a0fa12d4876a211d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:08.134944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af122bd30c3b42d9a0fa12d4876a211d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:08.139984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:08.222103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af122bd30c3b42d9a0fa12d4876a211d', 'request_id': 'req-8e9e4023-7e81-45af-a3f5-074e13035ea6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.231936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af122bd30c3b42d9a0fa12d4876a211d', 'request_id': 'req-8e9e4023-7e81-45af-a3f5-074e13035ea6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.241383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af122bd30c3b42d9a0fa12d4876a211d', 'request_id': 'req-8e9e4023-7e81-45af-a3f5-074e13035ea6'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.288868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] The token cache is being invalidate because project af122bd30c3b42d9a0fa12d4876a211d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:08.289320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.290135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e9e4023-7e81-45af-a3f5-074e13035ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.291880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1235/2430] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:08 2022] DELETE /identity/v3/projects/af122bd30c3b42d9a0fa12d4876a211d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:08.295924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:08.300319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:08.331782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b2bbb-2ae9-4166-a93b-3eacfe6f7ea0 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:08.407404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:08.407404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:08.407404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:08.438688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:08.439638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:08.440062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:08.440472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] PATH_INFO: `/v3/users/7874dfdac61540b9a03a5cb1c6b65b0d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:08.442917 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7874dfdac61540b9a03a5cb1c6b65b0d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:08.446681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:08.455461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] MFA Rules not processed for user `4f45aecdffb641d696b8c05d676f71aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:08.458609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:08.474252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:08.494172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:08.538110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7874dfdac61540b9a03a5cb1c6b65b0d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.563663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7874dfdac61540b9a03a5cb1c6b65b0d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.593427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7874dfdac61540b9a03a5cb1c6b65b0d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.604515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7874dfdac61540b9a03a5cb1c6b65b0d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.606859 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7874dfdac61540b9a03a5cb1c6b65b0d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:08.626085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1196/2431] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:08 2022] DELETE /identity/v3/users/7874dfdac61540b9a03a5cb1c6b65b0d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:08.626438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1236/2432] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:08.630454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:08.630454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:08.727853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b387efb-efdb-49f3-8e91-59aee581335e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:08.733978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:08.734918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:08.735298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:08.735673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:08.736435 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:08.738537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:08.782406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1197/2433] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:08 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:54:08.788493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:08.789988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:08.806280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03858eb4-636b-4c64-af49-c931d6bb9b43 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:08.931363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:08.932462 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:08.933044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:08.933384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:08.934520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:08.936935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:08.981123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1237/2434] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:08 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:08.982317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:08.994708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb87b9-f6ac-4eda-bf3f-dcf5e42a3c46 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.002203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.002825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.003107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.003372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.004022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.005790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.023332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1198/2435] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:09.024573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.043548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69329a-b19f-4f6f-ac5a-21026ff86d02 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.048697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.049546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.050056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.050654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] PATH_INFO: `/v3/projects/fede0fb10e54453f82a422511f9ba600/users/143d9cf64fde4cf88187800a98dc8438/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.074475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fede0fb10e54453f82a422511f9ba600, user_id=143d9cf64fde4cf88187800a98dc8438, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.077359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.100951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1238/2436] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:09 2022] PUT /identity/v3/projects/fede0fb10e54453f82a422511f9ba600/users/143d9cf64fde4cf88187800a98dc8438/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:54:09.103925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.124378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a7b73-cf9e-4511-a07f-495e074aafb3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.164522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.166622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:09.239972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.240595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.240909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.241180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.241843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.243462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.249502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd2d4a-7dc4-44c7-bf43-33626a0ad2e6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.256024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.256903 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.257245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.257726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] PATH_INFO: `/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.258942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7aeb95cfb54bed9e608d58322b6e9f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.260256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1199/2437] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:09.261326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.261456 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.275238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5119-8cd9-425a-bc7f-e062de4e66bf admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.281088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.281648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.281855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.282135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] PATH_INFO: `/v3/projects/fede0fb10e54453f82a422511f9ba600/users/143d9cf64fde4cf88187800a98dc8438/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.282888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:09.289618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fede0fb10e54453f82a422511f9ba600, user_id=143d9cf64fde4cf88187800a98dc8438, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.295568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfebabc-62d0-426b-a692-3c569392b5e1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.300059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:09.322410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1200/2438] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:09 2022] PUT /identity/v3/projects/fede0fb10e54453f82a422511f9ba600/users/143d9cf64fde4cf88187800a98dc8438/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:09.345838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7aeb95cfb54bed9e608d58322b6e9f', 'request_id': 'req-22cdc4f4-3ea2-48f6-8aca-859efd892193'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:09.354939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7aeb95cfb54bed9e608d58322b6e9f', 'request_id': 'req-22cdc4f4-3ea2-48f6-8aca-859efd892193'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:09.369910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7aeb95cfb54bed9e608d58322b6e9f', 'request_id': 'req-22cdc4f4-3ea2-48f6-8aca-859efd892193'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:09.391444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] The token cache is being invalidate because project ca7aeb95cfb54bed9e608d58322b6e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:09.391813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:09.392572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-22cdc4f4-3ea2-48f6-8aca-859efd892193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:09.394435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1239/2439] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:09 2022] DELETE /identity/v3/projects/ca7aeb95cfb54bed9e608d58322b6e9f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:09.399810 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.400188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.400625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.431193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:09.433364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:09.517695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1201/2440] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:09.527724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.528409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.528964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.584900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:09.591634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:09.626354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1240/2441] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:09.628112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.641332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e5589-1b35-45a2-8ed6-769ac5894d34 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.648098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.648758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.649084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.649361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.650011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:54:09.650359 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.652329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.676025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1202/2442] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:54:09.677605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.690944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372834f-fee7-46b7-abaa-1aa8c4b06c0d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.698638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.699303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.699872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.700163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.700978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.703077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.764653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1241/2443] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:09 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:09.766936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.796378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1fc0-8774-4db1-b46c-eaec73f52d1d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.905038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.905655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.905933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.906204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.907084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.908943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:09.939365 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1203/2444] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:09 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:09.942692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:09.954100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0620c5-64ca-46e2-bcf0-1f6b0f8695a7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:09.971622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:09.972623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:09.974241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:09.974741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:09.975631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:09.979175 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:10.000302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:10.003436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1242/2445] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:10.016423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe8962-797d-4a66-8153-215f3c038e90 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:10.024623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:10.025249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.025572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.025972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.027257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:10.030341 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215762b7-9710-4b9f-9bc6-0d98ea8b3f5a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:10.048670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1204/2446] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:10.050118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:10.061206 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d065a-179a-4e02-8aa3-98bef433a142 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:10.067762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:10.068816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.069301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.069832 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] PATH_INFO: `/v3/projects/2c5ca93260864a198f3d58181699279d/users/6c4756f70dbf4a28988aa8822b3f8683/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.091313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5ca93260864a198f3d58181699279d, user_id=6c4756f70dbf4a28988aa8822b3f8683, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:10.096109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:10.114831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1243/2447] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:10 2022] PUT /identity/v3/projects/2c5ca93260864a198f3d58181699279d/users/6c4756f70dbf4a28988aa8822b3f8683/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:54:10.134790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.135081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.135371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.176342 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] MFA Rules not processed for user `6c4756f70dbf4a28988aa8822b3f8683`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:10.177555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:10.236367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:10.262907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1205/2448] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:10.273143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:10.274894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:10.348346 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a7da5-b1cf-422e-9cd2-f539407f66aa admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:10.352321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:10.352980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.354324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.354324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.364739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:10.366343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b6da-8f80-4a15-88e8-379f8d8f73e0 service glance] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:10.380044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1244/2449] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:10.498611 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:10.500035 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:10.643397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20f3c3-7194-475a-9b33-a1b237c75c64 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:10.650557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:10.651298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.651651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.652070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] PATH_INFO: `/v3/users/98ca2bd265d5456786cecc6d37a427df` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.654087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca2bd265d5456786cecc6d37a427df)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:10.656286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:10.737006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca2bd265d5456786cecc6d37a427df'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:10.752799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca2bd265d5456786cecc6d37a427df'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:10.769913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca2bd265d5456786cecc6d37a427df'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:10.780343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca2bd265d5456786cecc6d37a427df'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:10.781090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca2bd265d5456786cecc6d37a427df'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:10.814479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1206/2450] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:10 2022] DELETE /identity/v3/users/98ca2bd265d5456786cecc6d37a427df => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:10.816084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.816419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.816725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.828657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:10.830669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:10.857076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:10.858700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:10.970694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff6a33-2b1f-4c32-aa70-41a4f5787a54 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:10.991736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:10.992490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:10.992871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:10.993390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] PATH_INFO: `/v3/users/d0d8bda4641d4003b38c6661e152d9f6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:10.994766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d8bda4641d4003b38c6661e152d9f6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:10.997392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:11.021461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:11.091781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d8bda4641d4003b38c6661e152d9f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:11.108044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d8bda4641d4003b38c6661e152d9f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:11.138269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d8bda4641d4003b38c6661e152d9f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:11.156985 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d8bda4641d4003b38c6661e152d9f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:11.157888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-7a118f69-d813-49f4-9174-36690cb349d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d8bda4641d4003b38c6661e152d9f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:11.170607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1207/2451] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:10 2022] DELETE /identity/v3/users/d0d8bda4641d4003b38c6661e152d9f6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:11.174798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:11.175275 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:11.175630 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:11.220149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:11.221502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:11.356513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:11.395585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1208/2452] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:11.419825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:11.429685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1245/2453] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:11.435163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:11.442022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:11.443615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:11.519376 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa71666-8f10-48a4-ad43-1c198cba1cad None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:11.601709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:11.602513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:11.602888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:11.603250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:11.604012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14555eaf-349f-4e90-aa47-faa82a8300d8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:11.610834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:11.611868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:11.611998 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:11.612249 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:11.619592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:11.622666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:11.629067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:11.633042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:11.635062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1246/2454] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:54:11.649924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1209/2455] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:12.549897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:12.552030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:12.579888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d94fb-0cd8-4731-80fe-9883a4319a6c service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:12.595429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:12.596440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:12.597333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:12.597849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] PATH_INFO: `/v3/projects/6076d331e7bc4a8da0255af513578ece` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:12.600571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6076d331e7bc4a8da0255af513578ece)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:12.604248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:12.618825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:12.620397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:12.662742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347f390-c9d8-47b2-b8bc-613915b3d2bb service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:12.678473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:12.679192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:12.679475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:12.679937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] PATH_INFO: `/v3/projects/62505cc4155a4e62b85ea34def321ffb` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:12.681274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62505cc4155a4e62b85ea34def321ffb)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:12.691504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:12.743215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6076d331e7bc4a8da0255af513578ece', 'request_id': 'req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.754852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:12.759223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6076d331e7bc4a8da0255af513578ece', 'request_id': 'req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.782294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6076d331e7bc4a8da0255af513578ece', 'request_id': 'req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.792475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62505cc4155a4e62b85ea34def321ffb', 'request_id': 'req-f551e441-a689-46a2-8ed0-ae4b06b9e010'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.806645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] The token cache is being invalidate because project 6076d331e7bc4a8da0255af513578ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:12.807060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.808980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-acfbca9d-a8ec-4402-bb8c-edfbb1b85e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.811088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62505cc4155a4e62b85ea34def321ffb', 'request_id': 'req-f551e441-a689-46a2-8ed0-ae4b06b9e010'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.812993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1247/2456] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:12 2022] DELETE /identity/v3/projects/6076d331e7bc4a8da0255af513578ece => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:12.819961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101e30-36e4-48a2-ad2f-724eddf2cdf8 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:12.820250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101e30-36e4-48a2-ad2f-724eddf2cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:12.820543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101e30-36e4-48a2-ad2f-724eddf2cdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:12.825475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62505cc4155a4e62b85ea34def321ffb', 'request_id': 'req-f551e441-a689-46a2-8ed0-ae4b06b9e010'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.852162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] The token cache is being invalidate because project 62505cc4155a4e62b85ea34def321ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:12.852470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.854610 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-f551e441-a689-46a2-8ed0-ae4b06b9e010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:12.859746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1210/2457] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:12 2022] DELETE /identity/v3/projects/62505cc4155a4e62b85ea34def321ffb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:54:12.860861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-21101e30-36e4-48a2-ad2f-724eddf2cdf8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:12.862110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-21101e30-36e4-48a2-ad2f-724eddf2cdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:12.865012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6218b5-af06-4071-98eb-10e7201b811f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:12.865300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6218b5-af06-4071-98eb-10e7201b811f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:12.865574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6218b5-af06-4071-98eb-10e7201b811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:12.918706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-2b6218b5-af06-4071-98eb-10e7201b811f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:12.919980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2b6218b5-af06-4071-98eb-10e7201b811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:12.922917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:13.046295 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1248/2458] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:13.053598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.053887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.054165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.085539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1211/2459] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:13.094340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.094811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.095134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.118392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:13.122301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:13.152115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1249/2460] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:13.159110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.194644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:13.194825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cc4c3-cb5a-40c2-adfb-d82e253164f2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.195265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:13.218974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1212/2461] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:13.228307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.228800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.229892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.230272 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.230644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.231400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:54:13.231808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.240484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736181d-0bed-4f6b-9794-57c157c93185 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.249739 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.264533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.273542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.274182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.292097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.308828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:54:13.308828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.309136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1250/2462] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:54:13.310670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.322205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.338240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a7617-1429-4c97-80b0-ae43c64edda3 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.347756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.349529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.349809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.350148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.350734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.352958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.380580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1213/2463] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:54:13.381778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.405280 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f8f57-9ecf-44c1-831c-47f96736dd2f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.433597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.435009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.437875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.438354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.439437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1251/2464] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:54:13.440453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.441470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.450048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.649815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cb507-28cc-4032-b49c-e636cf0e02b1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.664424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.665099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.665378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.665650 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.667192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.669258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.721399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1252/2465] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/users => generated 282 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:13.724339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.742093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ce8b4-4fda-450a-80e9-e2fe44ea2ff7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.753855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.754518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.754797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.755068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.755719 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.757406 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.800182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1253/2466] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:13.805946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.829851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda308c-6bc0-4961-9b80-d8f43d145004 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.841715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.841996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.842247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.842532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:13.843788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:13.845636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:13.859038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1254/2467] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:13.864940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.869189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1214/2468] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/projects => generated 381 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:13.871351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:13.883730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829d8c-0b8a-4410-bcfa-1ad3e3e82941 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:13.960224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:13.997704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:13.998399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:13.998660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:13.998933 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] PATH_INFO: `/v3/projects/7edcc5b3c93847d4a2adcbeb124ecc10/users/c53431761184454289345b00499bd7fc/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.021761 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7edcc5b3c93847d4a2adcbeb124ecc10, user_id=c53431761184454289345b00499bd7fc, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.027887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567e013-6066-43ea-bcb1-5e0374ae4ca2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.043742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d027-d62f-41f8-8f8f-960971dac19d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:14.056854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:14.057628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.058005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.058406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.059146 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.061216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.071200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1255/2469] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:13 2022] PUT /identity/v3/projects/7edcc5b3c93847d4a2adcbeb124ecc10/users/c53431761184454289345b00499bd7fc/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:14.082553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.082767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.083046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.126052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1215/2470] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:54:13 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:54:14.126590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:14.136526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:14.142357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3b687-9e7a-4659-9e18-09a5d444d453 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:14.160786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] MFA Rules not processed for user `c53431761184454289345b00499bd7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:14.162241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:14.336894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1256/2471] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:14.343194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:14.343841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.344118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.344385 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.344883 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:14.345072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.347316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:14.349867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.393723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1216/2472] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:14.398773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:14.421968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c158ef-a601-426e-8089-ab3793af2919 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:14.450657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bf533-5118-4fb7-bcb7-1b60c610d7a1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:14.455646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:14.456279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.457233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.457528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.458192 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.460235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:14.461176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.461687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.466399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d08cf-4085-4015-a052-ed8a775d7988 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.468811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.482035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.483843 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.497367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1257/2473] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:14.498631 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:14.498791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1217/2474] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:54:14.518632 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccf63e-e556-4ddb-8824-e112f5556ab5 service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:14.525578 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:14.527721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.528009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.528294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] PATH_INFO: `/v3/projects/725d386301d648de85fd6c4767250c58/users/b3d877f0c71f4e18854813780214ed6a/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.550008 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=725d386301d648de85fd6c4767250c58, user_id=b3d877f0c71f4e18854813780214ed6a, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:14.552801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:14.606884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1258/2475] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:54:14 2022] PUT /identity/v3/projects/725d386301d648de85fd6c4767250c58/users/b3d877f0c71f4e18854813780214ed6a/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:14.611838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710c522-d6bd-4331-b42d-6d0bf43d95d7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:14.612123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710c522-d6bd-4331-b42d-6d0bf43d95d7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:14.612399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710c522-d6bd-4331-b42d-6d0bf43d95d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:14.688251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d710c522-d6bd-4331-b42d-6d0bf43d95d7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:14.689954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d710c522-d6bd-4331-b42d-6d0bf43d95d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:14.846947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1218/2476] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:14.894403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:14.896938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:14.986423 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:15.006893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f301-e9e6-470d-a469-8b4910f4ca7e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:15.030035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:15.030510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:15.030787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:15.031100 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:15.057137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:15.058599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ada7f0-48f4-4e22-ad93-54bb5ce777e4 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:15.076379 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1259/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:15.538933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47286a9-f91b-48d6-aec9-c5171e0bc04c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:15.539356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47286a9-f91b-48d6-aec9-c5171e0bc04c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:15.539992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47286a9-f91b-48d6-aec9-c5171e0bc04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:15.571161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b47286a9-f91b-48d6-aec9-c5171e0bc04c None None] MFA Rules not processed for user `c93f22104a7b4a7caf0cc8b7180ffc55`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:15.572447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b47286a9-f91b-48d6-aec9-c5171e0bc04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:15.699867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1219/2478] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:16.966312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:16.966754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:16.967617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:17.030512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:17.030968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:17.031069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:17.041121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] MFA Rules not processed for user `143d9cf64fde4cf88187800a98dc8438`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:17.042510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:17.113513 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] MFA Rules not processed for user `b3d877f0c71f4e18854813780214ed6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:17.116005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:17.203971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:17.224374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1260/2479] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:17.270479 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:17.273490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:17.343708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1220/2480] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:17.373550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:17.374978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:17.505342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bedb70-58fe-4109-b865-b9f7ab7eb4a6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:17.517254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:17.517700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e25c30-351e-4be6-a920-4dbc38f0d5d9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:17.518593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:17.519503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:17.520557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:17.524241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:17.527194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:17.528562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:17.529863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:17.530866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:17.552542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:17.554645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20a842-6d4a-4774-bfaa-21840ee55929 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:17.582854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1221/2481] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:54:17.696811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:17.698643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94947883-39ba-4649-8168-bcfb050b302b service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:17.737946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1261/2482] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:54:19.510125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d214ef-c577-4418-bd00-6afb51ace27a None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:19.510589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d214ef-c577-4418-bd00-6afb51ace27a None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:19.510773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d214ef-c577-4418-bd00-6afb51ace27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:19.535428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-d3d214ef-c577-4418-bd00-6afb51ace27a None None] MFA Rules not processed for user `4f45aecdffb641d696b8c05d676f71aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:19.555475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d3d214ef-c577-4418-bd00-6afb51ace27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:19.570368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:19.571321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:19.571801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:19.584107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1222/2483] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:19.602317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:19.622425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] MFA Rules not processed for user `b3d877f0c71f4e18854813780214ed6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:19.623838 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:19.640301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1262/2484] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:22.221450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a52c3-c3f9-452d-8bd9-e7c8332e8671 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:22.222033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a52c3-c3f9-452d-8bd9-e7c8332e8671 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:22.222221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a52c3-c3f9-452d-8bd9-e7c8332e8671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:22.251252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f16a52c3-c3f9-452d-8bd9-e7c8332e8671 None None] MFA Rules not processed for user `49f80b68650940e8850cbd4f84a49ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:22.253382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f16a52c3-c3f9-452d-8bd9-e7c8332e8671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:22.365691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1223/2485] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:23.982826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:23.984325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:23.998136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5d294-791d-41e8-a99e-c4354227c93c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:24.002510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:24.003119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:24.003399 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:24.003671 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] PATH_INFO: `/v3/users/2653e7bc5e174152a7c18b3635d9b549` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:24.004944 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2653e7bc5e174152a7c18b3635d9b549)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:24.007645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:24.056038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2653e7bc5e174152a7c18b3635d9b549'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.062805 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2653e7bc5e174152a7c18b3635d9b549'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.074979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2653e7bc5e174152a7c18b3635d9b549'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.083291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2653e7bc5e174152a7c18b3635d9b549'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.083907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2653e7bc5e174152a7c18b3635d9b549'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.090041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1263/2486] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:23 2022] DELETE /identity/v3/users/2653e7bc5e174152a7c18b3635d9b549 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:24.094494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:24.095015 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:24.095112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:24.129310 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:24.130796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:24.203646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1224/2487] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:54:24.213816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:24.215260 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:24.288780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015da73-98d3-4054-8b73-5486a424ad98 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:24.299411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:24.300042 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:24.300324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:24.300621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:24.311055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:24.312535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:24.324783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1264/2488] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:24.661636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:24.663744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:24.683986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88a961-c3fa-4d1f-a935-59aa07f466d2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:24.690107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:24.690751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:24.691029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:24.692021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] PATH_INFO: `/v3/projects/e20f1eadf47949a5a51451bb06de2d41` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:24.693680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e20f1eadf47949a5a51451bb06de2d41)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:24.696903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:24.700334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:24.780496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e20f1eadf47949a5a51451bb06de2d41', 'request_id': 'req-9de91625-334d-4124-8dec-3693b96428b4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.804715 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e20f1eadf47949a5a51451bb06de2d41', 'request_id': 'req-9de91625-334d-4124-8dec-3693b96428b4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.810268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e20f1eadf47949a5a51451bb06de2d41', 'request_id': 'req-9de91625-334d-4124-8dec-3693b96428b4'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.835587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] The token cache is being invalidate because project e20f1eadf47949a5a51451bb06de2d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:24.836076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.837166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9de91625-334d-4124-8dec-3693b96428b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:24.862001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1225/2489] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:24 2022] DELETE /identity/v3/projects/e20f1eadf47949a5a51451bb06de2d41 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:34.318109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:34.837034 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:48.314608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:48.316064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:48.328545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137e3f4-132e-454d-b33d-57ef0b1badd2 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:48.387796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:48.388455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:48.388778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:48.389045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] PATH_INFO: `/v3/users/6c4756f70dbf4a28988aa8822b3f8683` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:48.390231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4756f70dbf4a28988aa8822b3f8683)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:48.392141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:48.456760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4756f70dbf4a28988aa8822b3f8683'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:48.468087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4756f70dbf4a28988aa8822b3f8683'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:48.481804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4756f70dbf4a28988aa8822b3f8683'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:48.492640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4756f70dbf4a28988aa8822b3f8683'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:48.492640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4756f70dbf4a28988aa8822b3f8683'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:48.502402 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1265/2490] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:54:48 2022] DELETE /identity/v3/users/6c4756f70dbf4a28988aa8822b3f8683 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:48.507155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:48.507624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:48.507830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:48.526893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:54:48.528183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:48.588363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1226/2491] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:54:48.602115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:48.604531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:48.665477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e65fbb-d4b3-4e79-8886-7ac0b2a74be9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:48.669263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:48.669900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:48.670180 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:48.670451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:48.679658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:48.681134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:48.693987 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1266/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:54:49.001438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:54:49.003533 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:54:49.021073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237a740-0ee2-44b7-9de0-4d8263ee9161 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:54:49.025825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:54:49.026472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:54:49.026918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:54:49.027204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] PATH_INFO: `/v3/projects/2c5ca93260864a198f3d58181699279d` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:54:49.028345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5ca93260864a198f3d58181699279d)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:54:49.030369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:54:49.083973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5ca93260864a198f3d58181699279d', 'request_id': 'req-01e69265-d144-4165-a056-37eeedbd35c3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:49.089382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5ca93260864a198f3d58181699279d', 'request_id': 'req-01e69265-d144-4165-a056-37eeedbd35c3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:49.099175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5ca93260864a198f3d58181699279d', 'request_id': 'req-01e69265-d144-4165-a056-37eeedbd35c3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:49.110842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] The token cache is being invalidate because project 2c5ca93260864a198f3d58181699279d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:54:49.111131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:49.111775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-01e69265-d144-4165-a056-37eeedbd35c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:54:49.113306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1227/2493] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:54:49 2022] DELETE /identity/v3/projects/2c5ca93260864a198f3d58181699279d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:54:58.686961 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:54:59.107657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:07.280808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:07.282218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:07.295259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d570b-b571-40b7-b712-2f0018f296c7 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:07.350207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:07.350831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:07.351107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:07.351378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] PATH_INFO: `/v3/users/c53431761184454289345b00499bd7fc` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:07.352485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53431761184454289345b00499bd7fc)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:07.354425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:07.399509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53431761184454289345b00499bd7fc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.406200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53431761184454289345b00499bd7fc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.418367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53431761184454289345b00499bd7fc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.428417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53431761184454289345b00499bd7fc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.429194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53431761184454289345b00499bd7fc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.435595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1267/2494] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:55:07 2022] DELETE /identity/v3/users/c53431761184454289345b00499bd7fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:07.440213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:07.440724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:07.440823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:07.459771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:07.460997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:07.522124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1228/2495] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:55:07.531802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:07.533870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:07.590840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eb0eb-de53-45b6-b628-0f902b757a6a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:07.594384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:07.594979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:07.595254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:07.595520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:07.604558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:07.606067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720cbba0-eba9-4137-bbcc-19fc5ca2cd8f service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:07.615303 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1268/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:55:07.881796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:07.883319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:07.896317 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70fef3-31a3-4ded-a1df-9c550cb421d5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:07.902197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:07.902833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:07.903106 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:07.903371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] PATH_INFO: `/v3/projects/7edcc5b3c93847d4a2adcbeb124ecc10` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:07.904490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7edcc5b3c93847d4a2adcbeb124ecc10)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:07.907563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:07.954966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7edcc5b3c93847d4a2adcbeb124ecc10', 'request_id': 'req-623575f5-4feb-472c-bc0b-0f4b6459e17f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.960390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7edcc5b3c93847d4a2adcbeb124ecc10', 'request_id': 'req-623575f5-4feb-472c-bc0b-0f4b6459e17f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.967778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7edcc5b3c93847d4a2adcbeb124ecc10', 'request_id': 'req-623575f5-4feb-472c-bc0b-0f4b6459e17f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.978942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] The token cache is being invalidate because project 7edcc5b3c93847d4a2adcbeb124ecc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:55:07.979231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.979855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-623575f5-4feb-472c-bc0b-0f4b6459e17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:07.981351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1229/2497] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:55:07 2022] DELETE /identity/v3/projects/7edcc5b3c93847d4a2adcbeb124ecc10 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:07.986750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:07.987033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:07.987302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.007155 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:08.008348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:08.024841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:08.067948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1269/2498] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:55:08.075713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.076036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.076315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.096531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:08.097781 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:08.105568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1230/2499] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:08.109563 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.119307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dead320-7ca8-43dd-94f4-3a91827988fc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.123738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.124317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.124593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.124887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.125512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:55:08.125811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.127980 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.139912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1270/2500] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:55:08.144553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.153962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8e4ce-c6f9-40f3-8c3e-32b95627922f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.158768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.159387 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.159667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.159937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.160623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.162544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.190322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1231/2501] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:55:08 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:55:08.195395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.205304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ec052-7fa1-4ae5-8e38-c9037855fcb0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.259534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.260129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.260438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.260750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.261381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.263276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.283791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1271/2502] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:55:08 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:55:08.287811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.297782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c236f-0d38-45ca-8e56-d56eb81e3ef3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.303228 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.303815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.304088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.304357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.305020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.306623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.316679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1232/2503] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:08.320309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.330003 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76957e3a-f8d9-4edf-8383-fb6c2fc87205 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.334790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.335375 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.335652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.335922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.336619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.338307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11e6e-9630-47fa-971d-ec236a2811e3 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.349262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1272/2504] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:08.355239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.365913 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978734c-9132-469e-b2d1-e473492d0ceb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.370526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.371172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.371449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.371723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] PATH_INFO: `/v3/projects/2bf8a8097f514bd5b31f5962183fb471/users/58589e0cbfe84948bcdfb823502ec98d/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.386055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf8a8097f514bd5b31f5962183fb471, user_id=58589e0cbfe84948bcdfb823502ec98d, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.388829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.401269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1233/2505] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:55:08 2022] PUT /identity/v3/projects/2bf8a8097f514bd5b31f5962183fb471/users/58589e0cbfe84948bcdfb823502ec98d/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:08.412082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81a0c-cd0e-42f2-be11-3b710305eb8e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.412368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81a0c-cd0e-42f2-be11-3b710305eb8e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.412678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c81a0c-cd0e-42f2-be11-3b710305eb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.440701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a1c81a0c-cd0e-42f2-be11-3b710305eb8e None None] MFA Rules not processed for user `58589e0cbfe84948bcdfb823502ec98d`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:08.441967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a1c81a0c-cd0e-42f2-be11-3b710305eb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:08.507271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1273/2506] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:08.517936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:08.519877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:08.541246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:08.579202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e97afd-3554-4fbf-ab52-212153482772 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:08.583403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:08.584000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:08.584276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:08.584542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:08.592908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:08.594334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:08.603528 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1234/2507] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:55:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:55:09.961543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da650e2a-70b5-4fa2-b8d0-f21c5442ae44 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:09.962029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da650e2a-70b5-4fa2-b8d0-f21c5442ae44 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:09.962132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da650e2a-70b5-4fa2-b8d0-f21c5442ae44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:09.983762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-da650e2a-70b5-4fa2-b8d0-f21c5442ae44 None None] MFA Rules not processed for user `58589e0cbfe84948bcdfb823502ec98d`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:09.985032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-da650e2a-70b5-4fa2-b8d0-f21c5442ae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:09.992552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1274/2508] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:10.955204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:10.956668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:11.024224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564d671-92ca-4e4b-a751-f38cf5c4c747 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:11.029282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:11.029912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:11.030191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:11.030489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:11.032195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:11.043494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:11.045101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:11.055305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1235/2509] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:55:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:55:19.978233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:21.051154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=49 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:41.608871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824aac0-a9d6-49c2-b94b-c624451f5b96 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:41.609939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824aac0-a9d6-49c2-b94b-c624451f5b96 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:41.610035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824aac0-a9d6-49c2-b94b-c624451f5b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:41.629357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4824aac0-a9d6-49c2-b94b-c624451f5b96 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:41.630549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4824aac0-a9d6-49c2-b94b-c624451f5b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:41.684223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1275/2510] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:55:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:41.727691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:41.729147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:41.799193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be62f2-dfb1-45b9-b7c3-a812f3a27139 service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:41.815177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:41.815866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:41.816065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:41.816454 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:41.825517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:41.827097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:41.836079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1236/2511] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:55:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:55:42.361924 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdbffe-0ac7-4c3e-a5d5-0ab3946b9d90 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:42.362255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdbffe-0ac7-4c3e-a5d5-0ab3946b9d90 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:42.362733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdbffe-0ac7-4c3e-a5d5-0ab3946b9d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:42.387284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-a5fdbffe-0ac7-4c3e-a5d5-0ab3946b9d90 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:42.388502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a5fdbffe-0ac7-4c3e-a5d5-0ab3946b9d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:42.396684 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1276/2512] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 04:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:42.409603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:42.420002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b38da-d758-4add-8970-f78e8ec14154 service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:42.425563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:42.426147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:42.426427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:42.426698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:42.434496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:42.436059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:42.444555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1237/2513] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:55:42.717230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6965-9142-4a3a-bc2f-ac8a2e6e020c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:42.717907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6965-9142-4a3a-bc2f-ac8a2e6e020c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:42.717907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6965-9142-4a3a-bc2f-ac8a2e6e020c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:42.738119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4e9d6965-9142-4a3a-bc2f-ac8a2e6e020c None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:42.739318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4e9d6965-9142-4a3a-bc2f-ac8a2e6e020c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:42.746368 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1277/2514] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:42.758822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:42.769958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cad868-4b06-4d16-bf50-50654b0242c2 service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:42.774966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:42.775547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:42.775826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:42.776095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:42.784472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:42.786066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:42.793855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1238/2515] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 04:55:43.115150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:43.115441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:43.115718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:43.134990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:43.136194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:43.144220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1278/2516] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 04:55:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:43.156272 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:43.166771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91440-a6c3-4f26-872d-2cdb6f0b116b service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:43.172179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:43.172796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:43.173107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:43.173376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:43.181127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:43.182713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127c4fb-6723-4583-9e00-87dc75ba3b1c service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:43.190975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1239/2517] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 04:55:46.309021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:46.369622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeee06-4b41-44ef-bf06-5622be021a64 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:46.373989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:46.375188 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:46.375464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:46.375730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] PATH_INFO: `/v3/users/58589e0cbfe84948bcdfb823502ec98d` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:46.376873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58589e0cbfe84948bcdfb823502ec98d)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:46.378862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:46.424657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58589e0cbfe84948bcdfb823502ec98d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.434439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58589e0cbfe84948bcdfb823502ec98d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.447916 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58589e0cbfe84948bcdfb823502ec98d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.455936 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58589e0cbfe84948bcdfb823502ec98d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.456757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58589e0cbfe84948bcdfb823502ec98d'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.464844 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1279/2518] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:55:46 2022] DELETE /identity/v3/users/58589e0cbfe84948bcdfb823502ec98d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:46.471271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:46.471735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:46.471891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:46.492830 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:46.494088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:46.557910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1240/2519] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:46.569443 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:46.571172 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:46.586321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:46.647948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d50e1-f809-4489-bad7-43e578661ef9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:46.653427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:46.654179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:46.654573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:46.654937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:46.665626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:46.667052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76cb6-bd4a-4d39-89fc-9b47d4601b3a service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:46.677153 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1280/2520] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:55:46.841531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:46.843007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:46.855566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d504178-6153-41ad-a105-612fbe49ca8c None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:46.861014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:46.861670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:46.861954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:46.862230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] PATH_INFO: `/v3/projects/2bf8a8097f514bd5b31f5962183fb471` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:46.863355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf8a8097f514bd5b31f5962183fb471)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:46.865433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:46.910036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:46.913328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf8a8097f514bd5b31f5962183fb471', 'request_id': 'req-ed20e892-1c35-4e28-9ca5-61176494cc8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.919519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf8a8097f514bd5b31f5962183fb471', 'request_id': 'req-ed20e892-1c35-4e28-9ca5-61176494cc8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.927616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf8a8097f514bd5b31f5962183fb471', 'request_id': 'req-ed20e892-1c35-4e28-9ca5-61176494cc8f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.938623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] The token cache is being invalidate because project 2bf8a8097f514bd5b31f5962183fb471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:55:46.938918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.939567 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ed20e892-1c35-4e28-9ca5-61176494cc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:55:46.941094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1241/2521] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:55:46 2022] DELETE /identity/v3/projects/2bf8a8097f514bd5b31f5962183fb471 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:46.945954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:46.946243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:46.946519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:46.966506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:46.967724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:47.025643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1281/2522] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:47.034823 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.035109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.035403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.054516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:47.055711 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:47.063517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1242/2523] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:47.067568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.077325 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d82cc-5b00-45b8-a8af-17ee27b7dbc9 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.082652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.083355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.083710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.084050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.084870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:55:47.085259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.086891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.099332 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1282/2524] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:55:47.103687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.112788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514381-000f-4d41-ad2e-fa78e002cc90 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.117441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.118029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.118308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.118578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.119205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.121163 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.152659 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1243/2525] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:55:47 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:55:47.157307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.169107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479a20f-3f58-41ad-b475-9b766d68b77c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.228110 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.228783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.229066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.229341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.229981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.231880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.252072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1283/2526] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:55:47 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:55:47.256013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.266577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65134b3d-0a66-4f37-b57c-f0761d6ad9f7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.271464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.272044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.272318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.272583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.273245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.274821 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.284962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1244/2527] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:47.288853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.298572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0c418-93a0-4387-8b79-978d7e51dba8 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.303159 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.303770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.304050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.304324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.304983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.306561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd1a56-6abb-4c57-8b03-8e732831199a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.317414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1284/2528] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:47.321181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.331346 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f0491f-4930-4da9-ada5-17c90ca7cf00 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.335929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.336510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.336819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.337088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] PATH_INFO: `/v3/projects/46ddf925b345471da94d05998a6a0540/users/946a0fd33f754ce1aa534ab88dc95aa9/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.351265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ddf925b345471da94d05998a6a0540, user_id=946a0fd33f754ce1aa534ab88dc95aa9, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.354014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.367409 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1245/2529] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:55:47 2022] PUT /identity/v3/projects/46ddf925b345471da94d05998a6a0540/users/946a0fd33f754ce1aa534ab88dc95aa9/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:55:47.372391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c184-0ae6-4767-bbc5-3363d8fe6567 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.372720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c184-0ae6-4767-bbc5-3363d8fe6567 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.373000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5c184-0ae6-4767-bbc5-3363d8fe6567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.392491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9ca5c184-0ae6-4767-bbc5-3363d8fe6567 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:47.393976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9ca5c184-0ae6-4767-bbc5-3363d8fe6567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:47.401947 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:47.452773 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1285/2530] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:47.462238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:47.463950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:47.520460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d89c5-b705-4e5c-8294-a29ae453d699 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:47.524073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:47.524695 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:47.524974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:47.525243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:47.534405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:47.535818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3a78c-3885-40fb-972e-64f81a8a6019 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:47.544891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1246/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:55:49.827665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:49.828185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:49.828652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:49.870318 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] MFA Rules not processed for user `b3d877f0c71f4e18854813780214ed6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:49.871656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:49.984741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1286/2532] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:53.819194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:53.820180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:53.820662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:53.892223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] MFA Rules not processed for user `946a0fd33f754ce1aa534ab88dc95aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:53.893131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:53.997657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1247/2533] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:55:54.001922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.016148 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e138bb5-e4b4-41c4-956a-a1c81a001701 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.022438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.023503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.024015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.024502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.025420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.027625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.121997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.123756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1287/2534] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:55:54 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:55:54.134502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:54.168150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4644bb-29db-493f-bf48-8d4250f0e725 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.251287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.252350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.252893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.253384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.254554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.258372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.293597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1248/2535] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:55:54 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:55:54.297594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.307979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed3011-1bb8-4fcb-9422-0356ea7e7b03 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.313963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.314803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.315262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.315731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.316565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.318418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.330145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1288/2536] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:54.334858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.350495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7325c2c-2857-4dec-9b0c-9b39cd6fd8a8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.357635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.358729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.359925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.360447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] PATH_INFO: `/v3/projects/1f97be6845804bd48b814faa1e32b34f/users/a3081ea34e774ac89302eee4bff2cdc9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.384642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f97be6845804bd48b814faa1e32b34f, user_id=a3081ea34e774ac89302eee4bff2cdc9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.387874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.411508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1249/2537] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:55:54 2022] PUT /identity/v3/projects/1f97be6845804bd48b814faa1e32b34f/users/a3081ea34e774ac89302eee4bff2cdc9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:55:54.413481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.425066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98709f3-b0f3-4daf-acea-c91e9518ef73 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.514134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.514943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.515351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.515628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.516274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.518179 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd0c81-2168-4bb7-b6fd-b0ba3c67a403 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.533934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1289/2538] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:55:54.535856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:55:54.549587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0d66a-a11d-4429-8387-494f675451d6 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:55:54.555404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:55:54.556671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:54.557250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:54.557806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] PATH_INFO: `/v3/projects/1f97be6845804bd48b814faa1e32b34f/users/a3081ea34e774ac89302eee4bff2cdc9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:54.560517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f97be6845804bd48b814faa1e32b34f, user_id=a3081ea34e774ac89302eee4bff2cdc9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:55:54.565100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609aa57-6d45-450b-8256-91d098c61f33 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:55:54.589036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1250/2539] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:55:54 2022] PUT /identity/v3/projects/1f97be6845804bd48b814faa1e32b34f/users/a3081ea34e774ac89302eee4bff2cdc9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:55:57.896989 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:55:57.897698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:55:57.898956 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:55:57.944786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] MFA Rules not processed for user `c93f22104a7b4a7caf0cc8b7180ffc55`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:55:57.949948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:55:57.987995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:55:58.079500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1290/2540] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:55:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:56:01.217846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:01.218339 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:01.218458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:01.251620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] MFA Rules not processed for user `a3081ea34e774ac89302eee4bff2cdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:56:01.252822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:01.346118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1251/2541] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:56:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:56:01.551418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:01.552982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:01.670507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186f5f-c988-4e4e-8dae-61427436c942 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:01.676978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:01.677588 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:01.677862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:01.678131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:01.752152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:01.754311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:01.767648 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1291/2542] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:56:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:56:10.267054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:10.268815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:10.316639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:10.342109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f74a62-f943-4414-9035-0ef3804142d0 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:10.346645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:10.347276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:10.347559 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:10.347833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] PATH_INFO: `/v3/users/4f45aecdffb641d696b8c05d676f71aa` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:10.348978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f45aecdffb641d696b8c05d676f71aa)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:10.350900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:10.401459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f45aecdffb641d696b8c05d676f71aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.409047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f45aecdffb641d696b8c05d676f71aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.428932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f45aecdffb641d696b8c05d676f71aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.443889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f45aecdffb641d696b8c05d676f71aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.444622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f45aecdffb641d696b8c05d676f71aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.451115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1252/2543] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:56:10 2022] DELETE /identity/v3/users/4f45aecdffb641d696b8c05d676f71aa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:10.676904 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:10.689568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35c33b-8e91-4770-9427-63a76fc29bda service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:10.750901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:10.751525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:10.751816 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:10.752066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] PATH_INFO: `/v3/projects/2e65f62fae68435ea0304bebc2a21b90` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:10.753190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e65f62fae68435ea0304bebc2a21b90)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:10.755067 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:10.761754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:10.808220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e65f62fae68435ea0304bebc2a21b90', 'request_id': 'req-a07df258-9a0e-441d-966f-df870933d18e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.816849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e65f62fae68435ea0304bebc2a21b90', 'request_id': 'req-a07df258-9a0e-441d-966f-df870933d18e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.825253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e65f62fae68435ea0304bebc2a21b90', 'request_id': 'req-a07df258-9a0e-441d-966f-df870933d18e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.840546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] The token cache is being invalidate because project 2e65f62fae68435ea0304bebc2a21b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:56:10.840880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.841506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.843463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1292/2544] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:56:10 2022] DELETE /identity/v3/projects/2e65f62fae68435ea0304bebc2a21b90 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:10.847470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:10.849483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:10.864135 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec88e9c-64fd-42c7-b47d-f3261aeb7a12 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:10.929451 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:10.930375 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:10.930784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:10.931184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] PATH_INFO: `/v3/users/143d9cf64fde4cf88187800a98dc8438` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:10.932858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=143d9cf64fde4cf88187800a98dc8438)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:10.936234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:10.992333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143d9cf64fde4cf88187800a98dc8438'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:10.999125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143d9cf64fde4cf88187800a98dc8438'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.003416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:11.013173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143d9cf64fde4cf88187800a98dc8438'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.021182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143d9cf64fde4cf88187800a98dc8438'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.021784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2ec6d1ef-7926-42ba-b687-89e58a5533ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143d9cf64fde4cf88187800a98dc8438'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.027589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1253/2545] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:56:10 2022] DELETE /identity/v3/users/143d9cf64fde4cf88187800a98dc8438 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:11.206534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:11.208073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:11.223010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07df258-9a0e-441d-966f-df870933d18e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:11.286767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:11.287424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:11.287703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:11.287975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] PATH_INFO: `/v3/projects/fede0fb10e54453f82a422511f9ba600` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:11.289185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fede0fb10e54453f82a422511f9ba600)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:11.291095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:11.337457 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fede0fb10e54453f82a422511f9ba600', 'request_id': 'req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.343485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fede0fb10e54453f82a422511f9ba600', 'request_id': 'req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.349493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fede0fb10e54453f82a422511f9ba600', 'request_id': 'req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.366408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] The token cache is being invalidate because project fede0fb10e54453f82a422511f9ba600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:56:11.366701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.367331 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d5367e3b-9a91-4a03-92d8-75b8b93e4d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:11.369119 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1293/2546] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:56:11 2022] DELETE /identity/v3/projects/fede0fb10e54453f82a422511f9ba600 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:21.024067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:21.363137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:24.196355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:24.196975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:24.196975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:24.202078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:24.202551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:24.202659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:24.217362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] MFA Rules not processed for user `946a0fd33f754ce1aa534ab88dc95aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:56:24.218565 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:24.221680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] MFA Rules not processed for user `49f80b68650940e8850cbd4f84a49ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:56:24.222853 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:24.275101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1254/2547] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:56:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:56:24.282950 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1294/2548] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:56:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:56:34.268201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=25 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:34.271633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=25 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:47.983716 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:47.985202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:48.042870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27889692-e06a-4b0d-9af9-3a248c7780f1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:48.046955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:48.047542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:48.047816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:48.048080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:48.049752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:48.110287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:48.111736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:48.121171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1255/2549] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:56:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:56:55.957059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:55.958997 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:56.025675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05012b18-5da8-477f-b558-aa742b2c5a0c None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:56.030894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:56.031538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:56.031840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:56.032147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] PATH_INFO: `/v3/users/b3d877f0c71f4e18854813780214ed6a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:56.033340 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d877f0c71f4e18854813780214ed6a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:56.035365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:56.081106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d877f0c71f4e18854813780214ed6a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.087807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d877f0c71f4e18854813780214ed6a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.100887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d877f0c71f4e18854813780214ed6a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.108096 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d877f0c71f4e18854813780214ed6a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.108784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d877f0c71f4e18854813780214ed6a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.114560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1295/2550] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:56:55 2022] DELETE /identity/v3/users/b3d877f0c71f4e18854813780214ed6a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:56.284760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:56.295896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df034e05-5960-4bbf-8f1a-4532d52cdfee service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:56.354506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:56.355151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:56.355438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:56.355712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] PATH_INFO: `/v3/projects/725d386301d648de85fd6c4767250c58` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:56.356883 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] RBAC: Authorizing `identity:delete_project(project_id=725d386301d648de85fd6c4767250c58)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:56.358826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:56.405682 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725d386301d648de85fd6c4767250c58', 'request_id': 'req-fc49f5f5-4468-458d-b00b-aafac011d63c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.407562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:56.412175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725d386301d648de85fd6c4767250c58', 'request_id': 'req-fc49f5f5-4468-458d-b00b-aafac011d63c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.419739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725d386301d648de85fd6c4767250c58', 'request_id': 'req-fc49f5f5-4468-458d-b00b-aafac011d63c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.430612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] The token cache is being invalidate because project 725d386301d648de85fd6c4767250c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:56:56.430901 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.431526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:56:56.433024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1256/2551] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:56:56 2022] DELETE /identity/v3/projects/725d386301d648de85fd6c4767250c58 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:56:59.269024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:56:59.270417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:59.330587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ae32-09e8-49b9-a6af-84a1e736e0b9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:56:59.363198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:56:59.382019 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:56:59.382622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:56:59.382900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:56:59.383194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:56:59.384937 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:56:59.456562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:56:59.457971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:56:59.470034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1296/2552] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:56:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:00.789612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:00.791001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:00.803072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49f5f5-4468-458d-b00b-aafac011d63c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:00.806606 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:00.807220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:00.807499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:00.807768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:00.809959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:00.868195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:00.869556 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:00.879009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1257/2553] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:09.132744 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:09.134167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:09.146852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29768-814b-475a-9228-3f74d21469db service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:09.150491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:09.151077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:09.151451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:09.151723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:09.153467 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:09.210965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:09.212383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7459153-d827-4c02-acf2-292f1cf98a36 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:09.221205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1297/2554] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:10.874858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=34,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:17.483612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:17.485809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:17.522501 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faee9e-9d4b-4963-8e21-8193dfcd7f33 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:17.532198 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:17.533225 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:17.534039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:17.534678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:17.540655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:17.557316 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:17.559245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:17.571702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1258/2555] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:19.179799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5df8-e47b-441c-8306-654d2d3151e1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:19.180330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5df8-e47b-441c-8306-654d2d3151e1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:19.180330 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d5df8-e47b-441c-8306-654d2d3151e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:19.201018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-443d5df8-e47b-441c-8306-654d2d3151e1 None None] MFA Rules not processed for user `c93f22104a7b4a7caf0cc8b7180ffc55`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:19.202239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-443d5df8-e47b-441c-8306-654d2d3151e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:19.209473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1298/2556] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:19.666394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:19.668978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:19.726733 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06249-6450-4b5c-a476-87e22f8fe2ad service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:19.730929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:19.730929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:19.731207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:19.731480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:19.733223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:19.743494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:19.745476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c857b-ba5c-4b0b-87c1-c9716bc42933 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:19.753965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1259/2557] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:57:20.086249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e8c-85fd-4c98-9619-f0ce820d37e2 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:20.086955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e8c-85fd-4c98-9619-f0ce820d37e2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:20.087369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4e8c-85fd-4c98-9619-f0ce820d37e2 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:20.089108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1299/2558] 10.222.0.27 () {54 vars in 961 bytes} [Tue Sep 20 04:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:57:20.095543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:20.095835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:20.096143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:20.119042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] MFA Rules not processed for user `097f3a7d28294befa98d30e220e01e83`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:20.121313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:20.189956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1260/2559] 10.222.0.27 () {58 vars in 1058 bytes} [Tue Sep 20 04:57:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:20.197959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4e8c-85fd-4c98-9619-f0ce820d37e2 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:20.211094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4e8c-85fd-4c98-9619-f0ce820d37e2 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:20.215929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:20.216505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:20.216804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:20.217075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:20.218751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:20.230444 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:20.232005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:20.241544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1300/2560] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:24.147011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:24.157511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5e14b-defb-4441-8931-a8bdcd67c9f9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:24.161105 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:24.161706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:24.161981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:24.162252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:24.163967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:24.173417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:24.174824 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:24.183056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1261/2561] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:25.387232 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:25.400208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c77fe5-986a-41c4-9ab0-464a5e13d34e service placement] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:25.406613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:25.407355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:25.407724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:25.407968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:25.409958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:25.423412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:25.425071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:25.437069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1301/2562] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:30.922129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:30.923535 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:30.934310 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c91597-f7e3-4778-ad3d-37e6a6c5ce30 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:30.937938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:30.938524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:30.938803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:30.939074 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:30.940793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:31.003117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:31.004548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:31.014666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1262/2563] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:34.309425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:34.311023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:34.324273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=82 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:34.332635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9cbf6-cab0-4dd9-8779-a3ae75bb647e service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:34.338647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:34.339288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.339742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.340017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] PATH_INFO: `/v3/users/49f80b68650940e8850cbd4f84a49ef4` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.341224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f80b68650940e8850cbd4f84a49ef4)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:34.343305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:34.405826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f80b68650940e8850cbd4f84a49ef4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.414580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f80b68650940e8850cbd4f84a49ef4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.428456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f80b68650940e8850cbd4f84a49ef4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.445639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f80b68650940e8850cbd4f84a49ef4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.446270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-eb4059d9-7492-4250-ba3f-f534d1d603d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f80b68650940e8850cbd4f84a49ef4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.455899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1302/2564] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:57:34 2022] DELETE /identity/v3/users/49f80b68650940e8850cbd4f84a49ef4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:57:34.636279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:34.656476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f820c-4358-46df-857d-057b71becdfa service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:34.660297 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:34.718032 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:34.718601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.718884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.719160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] PATH_INFO: `/v3/projects/5f0a24f6f6994f45bd314d3009b0a572` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.721355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0a24f6f6994f45bd314d3009b0a572)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:34.723440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:34.772431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0a24f6f6994f45bd314d3009b0a572', 'request_id': 'req-800762e1-b101-49ce-ba55-0e0ca559e5a7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.778423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0a24f6f6994f45bd314d3009b0a572', 'request_id': 'req-800762e1-b101-49ce-ba55-0e0ca559e5a7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.786526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0a24f6f6994f45bd314d3009b0a572', 'request_id': 'req-800762e1-b101-49ce-ba55-0e0ca559e5a7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.798663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] The token cache is being invalidate because project 5f0a24f6f6994f45bd314d3009b0a572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:57:34.798956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.799599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-800762e1-b101-49ce-ba55-0e0ca559e5a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:57:34.801098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1263/2565] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:57:34 2022] DELETE /identity/v3/projects/5f0a24f6f6994f45bd314d3009b0a572 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:57:34.806024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.806339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.806605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.829783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:34.830982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:34.898054 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1303/2566] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:34.905797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.906080 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.906355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.927155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:34.928363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:34.936969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1264/2567] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:34.941227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:34.951220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe1486-ff1a-481d-822f-b0336207c361 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:34.956000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:34.956624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.956906 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.957171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.957811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:57:34.958109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:34.959696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:34.971750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1304/2568] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:57:34.976134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:34.986426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeb260-5c36-4978-872b-8fa2c8736838 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:34.991145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:34.991746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:34.992025 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:34.992298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:34.992964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:34.994939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.024689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1265/2569] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:57:34 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:57:35.029091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:35.039393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c4956-3669-4c98-ae9d-284875c9032a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:35.097253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:35.097872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.098147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.098416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.099057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:35.101090 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.110840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:35.126381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1305/2570] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:57:35 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:57:35.133500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:35.145209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1f542-14b9-4f70-82fb-4396e88a48c7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:35.150251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:35.150851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.151130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.151400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.152036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:35.153666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.163897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1266/2571] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:57:35.168299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:35.179695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c934bf3-2b08-4c6a-b48b-8c313e167f72 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:35.184967 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:35.185553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.185831 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.186104 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.186731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:35.188310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5e10c-012c-4c9b-917e-6f5f7fdd13ba admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.199629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1306/2572] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:57:35.203932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:35.213953 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f28cc-4e61-4602-bf94-07a533fcfa4f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:35.219081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:35.219723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.220000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.220270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] PATH_INFO: `/v3/projects/56047142d69c4c1cbfa9f6a278622332/users/32cbf88b156745ae9a59e8f795e8ef58/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.235073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56047142d69c4c1cbfa9f6a278622332, user_id=32cbf88b156745ae9a59e8f795e8ef58, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:35.237748 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.250683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1267/2573] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:57:35 2022] PUT /identity/v3/projects/56047142d69c4c1cbfa9f6a278622332/users/32cbf88b156745ae9a59e8f795e8ef58/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:57:35.255850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5fe3-a1be-4ec5-861e-7695a3064355 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.256161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5fe3-a1be-4ec5-861e-7695a3064355 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.256452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5fe3-a1be-4ec5-861e-7695a3064355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.276169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4fbb5fe3-a1be-4ec5-861e-7695a3064355 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:35.277344 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4fbb5fe3-a1be-4ec5-861e-7695a3064355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:35.334873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1307/2574] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:57:35.344872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:35.346549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:35.380332 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:35.403362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d61b3d-90fa-453f-934f-d8c8999a0a97 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:35.406958 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:35.407549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:35.407827 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:35.408095 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:35.416793 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:35.418207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:35.427221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1268/2575] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:39.308145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1af15-fa2d-47ed-b2df-c7b35108dd6b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:39.308703 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1af15-fa2d-47ed-b2df-c7b35108dd6b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:39.308889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1af15-fa2d-47ed-b2df-c7b35108dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:39.338577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7ce1af15-fa2d-47ed-b2df-c7b35108dd6b None None] MFA Rules not processed for user `32cbf88b156745ae9a59e8f795e8ef58`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:39.339854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7ce1af15-fa2d-47ed-b2df-c7b35108dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:39.407651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1308/2576] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:39.422660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:39.424813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:39.527577 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe07f0-936b-4a56-b44f-47c15ee0c1f7 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:39.533180 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:39.533890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:39.534235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:39.534575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:39.543022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:39.544631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:39.553968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1269/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:41.121708 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:41.122212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:41.122321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:41.142000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] MFA Rules not processed for user `32cbf88b156745ae9a59e8f795e8ef58`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:57:41.143218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:41.151468 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1309/2578] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:57:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:57:49.549713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=45 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:57:50.155239 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:57:50.167654 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bd2de-5621-48ea-9232-7b88f2c0a7d7 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:57:50.173425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:57:50.174116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:57:50.174467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:57:50.174808 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:57:50.176579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:57:50.188852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:57:50.190438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:57:50.201548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1270/2579] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:57:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:57:51.136059 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:INSERT=2,keystone:SELECT=69 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:00.197643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=6 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:27.475248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:27.485215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e51826-dfc6-4ba0-8f95-8e280b0e8082 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:27.488685 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:27.489270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:27.489544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:27.489813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:27.491466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:27.506127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:27.507544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:27.517451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1310/2580] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:30.656788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:30.659751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:30.673673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e9b05-0e2b-4c24-84c5-45b31833fdfa service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:30.678138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:30.678759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:30.679038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:30.679313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] PATH_INFO: `/v3/users/c93f22104a7b4a7caf0cc8b7180ffc55` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:30.680434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93f22104a7b4a7caf0cc8b7180ffc55)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:30.682449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:30.731147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93f22104a7b4a7caf0cc8b7180ffc55'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:30.738287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93f22104a7b4a7caf0cc8b7180ffc55'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:30.750767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93f22104a7b4a7caf0cc8b7180ffc55'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:30.760869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93f22104a7b4a7caf0cc8b7180ffc55'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:30.761492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-72a5db12-9872-44f8-a442-07e9cd7e8fd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93f22104a7b4a7caf0cc8b7180ffc55'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:30.768133 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1271/2581] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:30 2022] DELETE /identity/v3/users/c93f22104a7b4a7caf0cc8b7180ffc55 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:30.938653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:30.949535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab007-f2c9-4f5b-981c-30023c6fbb94 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.006451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.007080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.007360 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.007627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] PATH_INFO: `/v3/projects/4f0f0b8842ae496292c27a6ee1350d32` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.009355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0f0b8842ae496292c27a6ee1350d32)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.011463 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.061076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0f0b8842ae496292c27a6ee1350d32', 'request_id': 'req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:31.066733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0f0b8842ae496292c27a6ee1350d32', 'request_id': 'req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:31.073675 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0f0b8842ae496292c27a6ee1350d32', 'request_id': 'req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:31.085629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] The token cache is being invalidate because project 4f0f0b8842ae496292c27a6ee1350d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:31.085915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:31.086571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-658e5cb5-51c6-4c7f-aac5-f3dcb2edfc34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:31.088058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1311/2582] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:30 2022] DELETE /identity/v3/projects/4f0f0b8842ae496292c27a6ee1350d32 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:31.093007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.093293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.093568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.113762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:31.114980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:31.177714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1272/2583] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:31.186574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.186887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.187213 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.206599 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:31.207811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:31.215543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1312/2584] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:31.219815 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.231037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da68d7-4c17-4ff5-a1f4-1281c971e705 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.236232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.236865 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.237145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.237417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.238061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:31.238377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.239956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.255820 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1273/2585] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:31.257143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.266949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc2082-254e-47a4-82cd-3460cd31f772 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.271845 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.272425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.272731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.272994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.273634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.275623 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.302438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1313/2586] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:31.306762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.316446 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ba3-7d0d-421b-b7a1-5e1b0fc03630 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.375147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.375751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.376030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.376300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.377089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.378986 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.391656 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:31.422334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1274/2587] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/users => generated 302 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:31.424276 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.439901 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207097c-51e0-4f0e-adcb-6a2252c75ec5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.447471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.448089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.448365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.448697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.449347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.451009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.470298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1314/2588] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:31.471809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.482947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952544e4-4e0b-40ef-9c44-bd8d191c355f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.487899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.489082 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.489367 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.489642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.490285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.492020 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d06a0-c807-4132-aa24-2e8b9850caaa admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.503482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1275/2589] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:31.507673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.518403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3463ee-c8c7-48fc-84e1-68d30e5ebdf2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.522983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.523603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.523881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.524207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] PATH_INFO: `/v3/projects/b9bd1a57ceb64a5b8ee6d5c0eae4a220/users/6bd8966932024fb78a01ac228da9d59a/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.539106 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bd1a57ceb64a5b8ee6d5c0eae4a220, user_id=6bd8966932024fb78a01ac228da9d59a, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.541830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.557959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1315/2590] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:31 2022] PUT /identity/v3/projects/b9bd1a57ceb64a5b8ee6d5c0eae4a220/users/6bd8966932024fb78a01ac228da9d59a/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:31.568345 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.568671 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.568970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.598721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] MFA Rules not processed for user `6bd8966932024fb78a01ac228da9d59a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:31.600014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:31.665709 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1276/2591] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:31.670421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.684157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:31.733804 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8908948-590b-4a37-af4b-6d3093f9e507 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.738473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.743907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.744205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.744571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.749077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.749077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.775125 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1316/2592] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:31.779516 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.790313 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3657b-0eb0-4e5e-8687-156ae8328843 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.846175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.846770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.847046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.847315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.847943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.849894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.871515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1277/2593] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:31 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:31.875498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.885767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65075757-302b-4033-bece-a365e0342c25 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.890659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.891249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.891526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.891796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.892427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.894085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.904370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1317/2594] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:31.908680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.919343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682153-adf8-4768-99cd-3b3cef072324 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:31.924644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:31.925445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:31.925728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:31.925999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] PATH_INFO: `/v3/projects/82aa483a92404d3b8e70f88120d09cab/users/d297a391a3604091ae800d2c38374f1b/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:31.942071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=82aa483a92404d3b8e70f88120d09cab, user_id=d297a391a3604091ae800d2c38374f1b, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:31.944768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:31.958161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1278/2595] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:31 2022] PUT /identity/v3/projects/82aa483a92404d3b8e70f88120d09cab/users/d297a391a3604091ae800d2c38374f1b/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:58:31.962135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:31.972464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cecdd-d6d5-4bb8-8e35-07303019375f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:32.033132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:32.033741 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.034045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.034317 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.034943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:32.036524 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672792-0171-42ce-af06-bba4c04f0dd5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:32.046607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1318/2596] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:58:32.050832 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:32.061729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb042-c983-4885-a079-8e6268391d6f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:32.066386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:32.067042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.067323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.067600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] PATH_INFO: `/v3/projects/82aa483a92404d3b8e70f88120d09cab/users/d297a391a3604091ae800d2c38374f1b/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.069625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82aa483a92404d3b8e70f88120d09cab, user_id=d297a391a3604091ae800d2c38374f1b, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:32.072291 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:32.083262 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1279/2597] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:32 2022] PUT /identity/v3/projects/82aa483a92404d3b8e70f88120d09cab/users/d297a391a3604091ae800d2c38374f1b/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:32.094971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.095502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.096021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.126751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] MFA Rules not processed for user `d297a391a3604091ae800d2c38374f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:32.128022 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:32.201311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1319/2598] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:32.212841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:32.214544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:32.237331 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:32.272769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef6c1-2616-44d9-8107-1e0948797555 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:32.276912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:32.277504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.277782 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.278052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.286773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:32.288204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811182c-c528-454f-81b8-a9264a50b7cf service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:32.298125 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1280/2599] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:32.442818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:32.452304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:32.503714 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb4d8a-311d-4c14-8692-67008911bc4d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:32.508124 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:32.508767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.509046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.509314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] PATH_INFO: `/v3/users/6bd8966932024fb78a01ac228da9d59a` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.510397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd8966932024fb78a01ac228da9d59a)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:32.512281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:32.557661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd8966932024fb78a01ac228da9d59a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:32.564498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd8966932024fb78a01ac228da9d59a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:32.576707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd8966932024fb78a01ac228da9d59a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:32.583796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd8966932024fb78a01ac228da9d59a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:32.584412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd8966932024fb78a01ac228da9d59a'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:32.590747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1320/2600] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:32 2022] DELETE /identity/v3/users/6bd8966932024fb78a01ac228da9d59a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:32.595270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.595573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.595849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.615543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:32.616818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:32.677613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1281/2601] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:32.690935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:32.692491 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:32.749618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcf43-74d0-49f1-8774-c0317125bb4e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:32.753374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:32.753974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:32.754251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:32.754537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:32.763639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:32.765496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:32.775620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1321/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:33.036743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.038645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:33.051961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d857ce-ce53-48b0-b643-f99ba87cb644 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.056856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:33.057495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.057769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.058065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] PATH_INFO: `/v3/projects/b9bd1a57ceb64a5b8ee6d5c0eae4a220` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.059290 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bd1a57ceb64a5b8ee6d5c0eae4a220)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:33.061139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:33.106765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bd1a57ceb64a5b8ee6d5c0eae4a220', 'request_id': 'req-bb7a457e-c6f0-4f53-b867-eb294ae3388d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.112288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bd1a57ceb64a5b8ee6d5c0eae4a220', 'request_id': 'req-bb7a457e-c6f0-4f53-b867-eb294ae3388d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.119607 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bd1a57ceb64a5b8ee6d5c0eae4a220', 'request_id': 'req-bb7a457e-c6f0-4f53-b867-eb294ae3388d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.133581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] The token cache is being invalidate because project b9bd1a57ceb64a5b8ee6d5c0eae4a220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:33.133872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.135012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.136010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1282/2603] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:33 2022] DELETE /identity/v3/projects/b9bd1a57ceb64a5b8ee6d5c0eae4a220 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:33.140225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.150683 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3f954-9223-41fa-bc53-8f04f5559fbb service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.184123 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:33.210790 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:33.211410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.212185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.212450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] PATH_INFO: `/v3/users/d297a391a3604091ae800d2c38374f1b` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.213629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d297a391a3604091ae800d2c38374f1b)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:33.215482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:33.260890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd297a391a3604091ae800d2c38374f1b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.267968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd297a391a3604091ae800d2c38374f1b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.280077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd297a391a3604091ae800d2c38374f1b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.286525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd297a391a3604091ae800d2c38374f1b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.287216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-54f4dd71-db13-49c3-9cef-86fbefb69680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd297a391a3604091ae800d2c38374f1b'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.293822 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1322/2604] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:33 2022] DELETE /identity/v3/users/d297a391a3604091ae800d2c38374f1b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:33.570640 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.572629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:33.586098 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a457e-c6f0-4f53-b867-eb294ae3388d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.630056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:33.644930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:33.645564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.645845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.646114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] PATH_INFO: `/v3/projects/82aa483a92404d3b8e70f88120d09cab` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.647213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82aa483a92404d3b8e70f88120d09cab)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:33.649764 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:33.693574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82aa483a92404d3b8e70f88120d09cab', 'request_id': 'req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.699558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82aa483a92404d3b8e70f88120d09cab', 'request_id': 'req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.706845 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82aa483a92404d3b8e70f88120d09cab', 'request_id': 'req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.718273 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] The token cache is being invalidate because project 82aa483a92404d3b8e70f88120d09cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:33.718584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.719206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c55d3045-5ab9-4802-8e60-dbd1f4c5bc12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:33.720713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1283/2605] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:33 2022] DELETE /identity/v3/projects/82aa483a92404d3b8e70f88120d09cab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:33.725520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.725890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.726167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.746553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:33.747749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:33.807850 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1323/2606] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:33.815573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.815856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.816132 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.836001 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:33.837366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:33.845992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1284/2607] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:33.850315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.860310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d5dd8-2002-4cc8-9602-3182b897927d None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.865050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:33.866085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.866085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.866288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.866809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:33.867114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:33.869270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:33.881692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1324/2608] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:33.886401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.896235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8c9f6-c61c-4058-920b-8a7e31e83e76 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.901172 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:33.901774 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:33.902057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:33.902330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:33.903022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:33.904962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:33.933818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1285/2609] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:33 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:33.937954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:33.948424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b12108-759c-4df9-a1e5-e6df4bdab1cd admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:33.980579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:34.005266 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:34.005856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.006130 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.006422 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.007044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:34.009590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:34.029806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1325/2610] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:33 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:34.034322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:34.044629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c57992-21c3-4d58-b6b9-70ad57a5acfa admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:34.049162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:34.049754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.050036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.050309 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.050949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:34.052568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:34.062775 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1286/2611] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:34.066505 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:34.076777 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4a033-bf06-47be-8b83-43ea27ac9eb1 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:34.081228 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:34.081802 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.082077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.082343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.082969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:34.084531 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aa22a-e9b2-428c-b67d-8ae11a0e71bc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:34.094233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1326/2612] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:34.098118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:34.108363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6679b-c338-4d7b-a10c-c142b43c5458 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:34.112961 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:34.113600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.113884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.114161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] PATH_INFO: `/v3/projects/ea0602a838444f35a1c4f6804d8ece1f/users/eaf1ab9827cc4396af1c8f017dc40ae6/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.130010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0602a838444f35a1c4f6804d8ece1f, user_id=eaf1ab9827cc4396af1c8f017dc40ae6, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:34.132712 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:34.146265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1287/2613] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:34 2022] PUT /identity/v3/projects/ea0602a838444f35a1c4f6804d8ece1f/users/eaf1ab9827cc4396af1c8f017dc40ae6/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:34.151323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0c3f-c32e-4abf-b2d3-826190310874 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.151782 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0c3f-c32e-4abf-b2d3-826190310874 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.151882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0c3f-c32e-4abf-b2d3-826190310874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.171143 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-310d0c3f-c32e-4abf-b2d3-826190310874 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:34.172345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-310d0c3f-c32e-4abf-b2d3-826190310874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:34.229246 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1327/2614] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:34.238784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:34.240462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:34.298576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1019-0f7c-4796-91d0-6f62b48a096b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:34.302071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:34.302662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.302936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.303201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.311875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:34.313318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:34.322992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1288/2615] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:34.932401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:34.932730 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:34.933005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:34.962384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] MFA Rules not processed for user `eaf1ab9827cc4396af1c8f017dc40ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:34.964873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:35.037958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1328/2616] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:35.042992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.053138 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5153b4-e745-4104-9c8f-5ac86db63e8b service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.058552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.059160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.059562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.059902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.060621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.062621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.082482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:35.103620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1289/2617] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:35 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:35.104534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.116709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44c628-89a4-4b2a-ab00-6d9a8dde6cbc None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.171826 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:35.188692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.189329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.189602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.189869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.190500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.192527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.215231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1329/2618] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:35 2022] POST /identity/v3/users => generated 309 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:35.219052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.229765 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51b58c-3f36-488d-9a96-a4778843d607 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.234904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.235517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.235790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.236051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.236736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.238347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.249411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1290/2619] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:35.253767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.264483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849847a-26c5-4118-85f5-080bfc5fcad5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.269465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.270109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.270387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.270660 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] PATH_INFO: `/v3/projects/27b9c0094f234ce9afd26a79ddd2c803/users/9d03b0dcb8354b6ea4fae3ad1632a3e9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.285769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b9c0094f234ce9afd26a79ddd2c803, user_id=9d03b0dcb8354b6ea4fae3ad1632a3e9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.288392 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.301345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1330/2620] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:35 2022] PUT /identity/v3/projects/27b9c0094f234ce9afd26a79ddd2c803/users/9d03b0dcb8354b6ea4fae3ad1632a3e9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:35.305344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.315924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64d981-0681-4103-aeaf-d41a0975ea8b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.371261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.371784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.372064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.372337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.372994 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.374578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19823305-f5c2-46e6-aca1-cdaa452362d9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.383900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1291/2621] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:35.388327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:35.398637 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d2b12-c989-4e52-80f6-637f90b1161c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:35.403655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:35.404284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:35.404664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:35.404854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] PATH_INFO: `/v3/projects/27b9c0094f234ce9afd26a79ddd2c803/users/9d03b0dcb8354b6ea4fae3ad1632a3e9/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:35.406710 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b9c0094f234ce9afd26a79ddd2c803, user_id=9d03b0dcb8354b6ea4fae3ad1632a3e9, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:35.409351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:35.420005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1331/2622] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:35 2022] PUT /identity/v3/projects/27b9c0094f234ce9afd26a79ddd2c803/users/9d03b0dcb8354b6ea4fae3ad1632a3e9/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:36.054099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:36.054638 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:36.054753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:36.082951 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] MFA Rules not processed for user `9d03b0dcb8354b6ea4fae3ad1632a3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:36.084161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:36.155992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1292/2623] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:36.597861 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:36.599577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:36.676574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d6e69-0e97-4db5-9949-6b4534cfe196 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:36.681607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:36.682210 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:36.682485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:36.682750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:36.684421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:36.697076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:36.698724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:36.709389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1332/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:37.164169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:37.224170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341440e-a71f-4706-8290-a47cae1af649 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:37.229103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:37.229722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:37.229999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:37.230269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] PATH_INFO: `/v3/users/eaf1ab9827cc4396af1c8f017dc40ae6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:37.231357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf1ab9827cc4396af1c8f017dc40ae6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:37.233306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:37.237739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:37.279489 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf1ab9827cc4396af1c8f017dc40ae6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.286368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf1ab9827cc4396af1c8f017dc40ae6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.298485 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf1ab9827cc4396af1c8f017dc40ae6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.306993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf1ab9827cc4396af1c8f017dc40ae6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.307608 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf1ab9827cc4396af1c8f017dc40ae6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.313445 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1293/2625] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:37 2022] DELETE /identity/v3/users/eaf1ab9827cc4396af1c8f017dc40ae6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:58:37.485935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:37.498395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c6a0e-0612-48bf-ac35-3ed0380fd3f2 service placement] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:37.563398 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:37.564048 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:37.564328 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:37.564620 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] PATH_INFO: `/v3/projects/ea0602a838444f35a1c4f6804d8ece1f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:37.565750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea0602a838444f35a1c4f6804d8ece1f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:37.567636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:37.601586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:37.615897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea0602a838444f35a1c4f6804d8ece1f', 'request_id': 'req-81fbc481-16fe-4b88-9b2c-ce2670953e43'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.621305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea0602a838444f35a1c4f6804d8ece1f', 'request_id': 'req-81fbc481-16fe-4b88-9b2c-ce2670953e43'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.627226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea0602a838444f35a1c4f6804d8ece1f', 'request_id': 'req-81fbc481-16fe-4b88-9b2c-ce2670953e43'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.638919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] The token cache is being invalidate because project ea0602a838444f35a1c4f6804d8ece1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:37.639205 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.640073 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.641558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1333/2626] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:37 2022] DELETE /identity/v3/projects/ea0602a838444f35a1c4f6804d8ece1f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:37.645524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:37.646932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:37.662305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bce8-84e7-4ca3-b7c9-871c02e2704f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:37.717097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:37.717721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:37.718003 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:37.718281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] PATH_INFO: `/v3/users/9d03b0dcb8354b6ea4fae3ad1632a3e9` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:37.719398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d03b0dcb8354b6ea4fae3ad1632a3e9)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:37.721356 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:37.766134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d03b0dcb8354b6ea4fae3ad1632a3e9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.773013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d03b0dcb8354b6ea4fae3ad1632a3e9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.785362 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d03b0dcb8354b6ea4fae3ad1632a3e9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.793320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d03b0dcb8354b6ea4fae3ad1632a3e9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.794081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-94bfe0ca-f7eb-4f2e-8ba0-1434cd551c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d03b0dcb8354b6ea4fae3ad1632a3e9'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:37.799945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1294/2627] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:37 2022] DELETE /identity/v3/users/9d03b0dcb8354b6ea4fae3ad1632a3e9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:37.975763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:37.977293 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:37.989951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbc481-16fe-4b88-9b2c-ce2670953e43 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.046914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.047473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.047749 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.048020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] PATH_INFO: `/v3/projects/27b9c0094f234ce9afd26a79ddd2c803` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.049137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b9c0094f234ce9afd26a79ddd2c803)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.051041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.096821 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b9c0094f234ce9afd26a79ddd2c803', 'request_id': 'req-a636463c-6102-4052-9dea-3e6be53b4980'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:38.102288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b9c0094f234ce9afd26a79ddd2c803', 'request_id': 'req-a636463c-6102-4052-9dea-3e6be53b4980'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:38.108253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b9c0094f234ce9afd26a79ddd2c803', 'request_id': 'req-a636463c-6102-4052-9dea-3e6be53b4980'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:38.120235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] The token cache is being invalidate because project 27b9c0094f234ce9afd26a79ddd2c803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:38.120522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:38.121183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a636463c-6102-4052-9dea-3e6be53b4980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:38.122658 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1334/2628] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:37 2022] DELETE /identity/v3/projects/27b9c0094f234ce9afd26a79ddd2c803 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:38.127721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.128009 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.128340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.136924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:38.152209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:38.153443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:38.210221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1295/2629] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:38.218508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.218796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.219068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.238216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:38.239409 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:38.247639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1335/2630] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:38.251843 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.262244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a736c-d026-4d85-bb17-ec17876fb38f None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.267164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.267749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.268026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.268294 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.268969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:38.269266 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.270849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.282045 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1296/2631] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:38.286184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.295277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8e7f-0a35-4f91-8457-c23fb638eb0a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.299889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.300415 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.300731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.301005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.301635 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.303499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.330721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1336/2632] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:38.335021 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.344760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674ec85-fd94-4f2d-85dc-64affc52bf83 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.399891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.400475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.400783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.401053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.401666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.403555 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.431880 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1297/2633] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:38.436226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.447490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e797-ab38-4d6b-85e0-fafc857a172b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.452061 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.452682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.452959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.453227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.453851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.455429 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.465136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1337/2634] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:38.469224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.478934 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c386ac-a237-457d-ac7a-e87321654a7e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.483515 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.484103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.484382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.484689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.485329 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.486904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21b4f6-8f97-4d61-bc49-f84788705fd1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.497619 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1298/2635] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:38.501535 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.511894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42221031-871f-4a93-af86-2a9655f56cbb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.516813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.517456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.517760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.518031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] PATH_INFO: `/v3/projects/8c6181b70f4f4479af1f40ac137f1b9c/users/56a598e29bbe4c6fad483efe15b0a601/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.521093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:38.533976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6181b70f4f4479af1f40ac137f1b9c, user_id=56a598e29bbe4c6fad483efe15b0a601, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.536679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.549573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1338/2636] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:38 2022] PUT /identity/v3/projects/8c6181b70f4f4479af1f40ac137f1b9c/users/56a598e29bbe4c6fad483efe15b0a601/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:38.561328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.561825 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.562130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.590199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] MFA Rules not processed for user `56a598e29bbe4c6fad483efe15b0a601`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:38.591412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:38.651140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:38.656858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1299/2637] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:38.661499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.724425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98fa6b3-1f1b-4d1f-bd6f-a1cc14005eca admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.729767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.730361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.730672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.730908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.731542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.733450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.759678 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1339/2638] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:38.763629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.774623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c87064-2e27-45dc-8a59-baf2342958af None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.829301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.829855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.830137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.830440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.831088 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.833043 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.856252 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1300/2639] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:38 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:38.860256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.871025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d222141-ab23-4de2-8e68-73cb164f8987 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.875569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.876151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.876427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.876731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.877352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.878930 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.889217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1340/2640] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:38.893386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.903193 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e8f5e-fc63-4285-9cd4-3886267e9237 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:38.907776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:38.908411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:38.908721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:38.908995 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] PATH_INFO: `/v3/projects/16a28c26135742ec9e552f5de3d33b28/users/a3e5625c266a4e54a73ac3a889b039bd/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:38.923971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a28c26135742ec9e552f5de3d33b28, user_id=a3e5625c266a4e54a73ac3a889b039bd, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:38.926730 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:38.940622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1301/2641] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:38 2022] PUT /identity/v3/projects/16a28c26135742ec9e552f5de3d33b28/users/a3e5625c266a4e54a73ac3a889b039bd/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:38.944647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:38.954586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d4d0e-d53b-4f1c-96dd-9754ceff1226 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:39.009878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:39.010477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:39.010750 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:39.011017 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:39.011639 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:39.013256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc59e-773f-4e41-9cc8-fcadae8d2bae admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:39.023546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1341/2642] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:58:39.027984 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:39.038160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed049d-3212-4efa-a232-a2df7a071fd2 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:39.043340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:39.043983 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:39.044292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:39.044563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] PATH_INFO: `/v3/projects/16a28c26135742ec9e552f5de3d33b28/users/a3e5625c266a4e54a73ac3a889b039bd/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:39.046475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a28c26135742ec9e552f5de3d33b28, user_id=a3e5625c266a4e54a73ac3a889b039bd, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:39.049137 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:39.059281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1302/2643] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:39 2022] PUT /identity/v3/projects/16a28c26135742ec9e552f5de3d33b28/users/a3e5625c266a4e54a73ac3a889b039bd/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:39.070634 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:39.070921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:39.071189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:39.099945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] MFA Rules not processed for user `a3e5625c266a4e54a73ac3a889b039bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:39.101098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:39.150540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:39.176952 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1342/2644] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:39.188044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:39.189874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:39.246191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130aaa3-f3d7-45d8-a6d5-b1adc2452e4d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:39.250230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:39.250888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:39.251170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:39.251443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:39.260504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:39.262044 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:39.271692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1303/2645] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:39.347862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:39.358014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b7cf1-e2b0-46b9-8af2-4817896446b6 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:39.361510 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:39.362097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:39.362481 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:39.362740 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:39.420263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:39.421711 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26521d22-7002-4f79-b047-1c58786071f7 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:39.430796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1343/2646] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:48.278726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:48.329320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:48.341928 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a115772-cc87-4566-82ab-1be8c700cfb9 service cinder] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:48.346819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:48.347720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:48.348004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:48.348274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] PATH_INFO: `/v3/users/56a598e29bbe4c6fad483efe15b0a601` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:48.349438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a598e29bbe4c6fad483efe15b0a601)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:48.351371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:48.406506 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a598e29bbe4c6fad483efe15b0a601'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.413863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a598e29bbe4c6fad483efe15b0a601'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.427706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a598e29bbe4c6fad483efe15b0a601'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.439029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a598e29bbe4c6fad483efe15b0a601'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.439795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a598e29bbe4c6fad483efe15b0a601'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.450842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1304/2647] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:48 2022] DELETE /identity/v3/users/56a598e29bbe4c6fad483efe15b0a601 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:48.455715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1c64-d878-4875-a38c-364ad426189a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:48.456187 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1c64-d878-4875-a38c-364ad426189a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:48.456408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1c64-d878-4875-a38c-364ad426189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:48.475692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-935a1c64-d878-4875-a38c-364ad426189a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:48.476983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-935a1c64-d878-4875-a38c-364ad426189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:48.538197 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1344/2648] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:48.549966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:48.554099 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:48.619945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6b346-d6ad-458e-a4a4-3b9c6bd39dd5 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:48.623745 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:48.624350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:48.624660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:48.624935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:48.634593 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:48.636036 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:48.646188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1305/2649] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:48.921055 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1c64-d878-4875-a38c-364ad426189a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:48.922642 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-935a1c64-d878-4875-a38c-364ad426189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:48.935649 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1c64-d878-4875-a38c-364ad426189a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:48.940209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:48.940881 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:48.941163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:48.941434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] PATH_INFO: `/v3/projects/8c6181b70f4f4479af1f40ac137f1b9c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:48.943056 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6181b70f4f4479af1f40ac137f1b9c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:48.945176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:48.992077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6181b70f4f4479af1f40ac137f1b9c', 'request_id': 'req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:48.998105 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6181b70f4f4479af1f40ac137f1b9c', 'request_id': 'req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.004087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6181b70f4f4479af1f40ac137f1b9c', 'request_id': 'req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.016970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] The token cache is being invalidate because project 8c6181b70f4f4479af1f40ac137f1b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:49.017256 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.017880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.019345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1345/2650] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:48 2022] DELETE /identity/v3/projects/8c6181b70f4f4479af1f40ac137f1b9c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:49.023285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:49.032705 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979748a2-b796-427d-9fed-10ec966b8d78 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:49.087108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:49.087744 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.088007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.088282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] PATH_INFO: `/v3/users/a3e5625c266a4e54a73ac3a889b039bd` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.089420 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e5625c266a4e54a73ac3a889b039bd)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:49.091323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:49.107574 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:49.137686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e5625c266a4e54a73ac3a889b039bd'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.144952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e5625c266a4e54a73ac3a889b039bd'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.158279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e5625c266a4e54a73ac3a889b039bd'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.166000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e5625c266a4e54a73ac3a889b039bd'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.166635 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-c5225ba6-24ef-4bff-a1f5-b1cf87c57904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e5625c266a4e54a73ac3a889b039bd'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.172405 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1306/2651] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:49 2022] DELETE /identity/v3/users/a3e5625c266a4e54a73ac3a889b039bd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:49.527590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:49.529167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:49.537702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:49.545767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9572e-b3ca-415f-8baa-e126a1c96bf4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:49.609238 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:49.609880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.610157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.610425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] PATH_INFO: `/v3/projects/16a28c26135742ec9e552f5de3d33b28` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.611617 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a28c26135742ec9e552f5de3d33b28)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:49.613653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:49.662502 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a28c26135742ec9e552f5de3d33b28', 'request_id': 'req-d25ec198-ee18-493d-b232-e2e02c0c2969'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.668655 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a28c26135742ec9e552f5de3d33b28', 'request_id': 'req-d25ec198-ee18-493d-b232-e2e02c0c2969'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.675618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a28c26135742ec9e552f5de3d33b28', 'request_id': 'req-d25ec198-ee18-493d-b232-e2e02c0c2969'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.688687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] The token cache is being invalidate because project 16a28c26135742ec9e552f5de3d33b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:49.688973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.689605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d25ec198-ee18-493d-b232-e2e02c0c2969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:49.691109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1346/2652] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:49 2022] DELETE /identity/v3/projects/16a28c26135742ec9e552f5de3d33b28 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:49.695918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.696406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.696508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.716231 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:49.717382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:49.774327 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1307/2653] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:49.781982 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.782273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.782544 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.802307 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:49.803489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:49.811262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1347/2654] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:49.815281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:49.825250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cebc71-1208-44f2-aa08-b2ab2ce72efe None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:49.829761 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:49.830348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.830625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.830892 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.831524 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:49.831818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:49.833433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:49.846440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1308/2655] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:49.850971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:49.860099 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319dab9-a411-4024-becf-271ea4a56895 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:49.864781 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:49.865365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.865640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.865907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.866526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:49.868428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:49.898194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1348/2656] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:49 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:49.902597 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:49.913254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731a1a5-84b8-482d-901c-6835d9db4c36 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:49.979558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:49.980393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:49.980949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:49.981234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:49.981937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:49.983912 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.013448 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1309/2657] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:49 2022] POST /identity/v3/users => generated 282 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:50.014968 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.026053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff87c-279c-4024-a278-cee2a3f30d64 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.030755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.031358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.031633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.031908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.032540 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.034171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.047240 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1349/2658] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:50.048575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.059085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b2ab9-b042-4c20-807d-053ececa8a5e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.064056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.065575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.065955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.066466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.067641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.068957 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755bf60-88ed-4308-8ee5-2967d2013cfd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.076584 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:50.082376 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1310/2659] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:58:50.088453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.100445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab05d37-1e01-4a24-baad-1f318eab4c04 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.105207 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.105960 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.106326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.106663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] PATH_INFO: `/v3/projects/580c8efe91cc444aaf1fe0a1ab724c20/users/3948fbe798554fa8a82634f91eb6edca/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.122674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580c8efe91cc444aaf1fe0a1ab724c20, user_id=3948fbe798554fa8a82634f91eb6edca, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.125610 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.148147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1350/2660] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:50 2022] PUT /identity/v3/projects/580c8efe91cc444aaf1fe0a1ab724c20/users/3948fbe798554fa8a82634f91eb6edca/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:58:50.160109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b015b-1927-45f1-94b4-00067a675ee3 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.160571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b015b-1927-45f1-94b4-00067a675ee3 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.161062 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b015b-1927-45f1-94b4-00067a675ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.191809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ff8b015b-1927-45f1-94b4-00067a675ee3 None None] MFA Rules not processed for user `3948fbe798554fa8a82634f91eb6edca`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:50.193069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ff8b015b-1927-45f1-94b4-00067a675ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:50.265877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1311/2661] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:50.270627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.300690 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.300979 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.301254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.308111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:50.322666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:50.323959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:50.346326 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae279f0-18e8-4850-b6b0-82224f02a9c2 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.351830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.352440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.352762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.353031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.353661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.355601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.388255 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1351/2662] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:50 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:50.392760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.394473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1312/2663] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:58:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:50.472864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6804c-df58-4d22-bacb-79e2418c78cc admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.479098 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.479817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.480051 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.480401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.481101 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.484577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.508521 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1352/2664] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:50 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:58:50.512625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.531605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.532915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:50.563964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9935e-1b7c-4b50-a00e-b2a1f9783170 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.583645 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.587935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.588502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.589472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.590938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.595536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.614576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ed244-6674-4d08-8ff9-20114d78a53f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.619598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1313/2665] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:58:50.621121 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.622052 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.622404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.622847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.624157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.651042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20582d6-de70-4439-8609-5ba25165e80c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.661379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.663684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.663976 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.664330 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] PATH_INFO: `/v3/projects/671c713c650c4b4cb562648fa367af37/users/3305a5ea28fe49369a8afdd55950ac18/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.679291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:50.693271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.695193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.705724 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] RBAC: Authorizing `identity:create_grant(project_id=671c713c650c4b4cb562648fa367af37, user_id=3305a5ea28fe49369a8afdd55950ac18, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.708434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.723050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1353/2666] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 04:58:50.739680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1314/2667] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:50 2022] PUT /identity/v3/projects/671c713c650c4b4cb562648fa367af37/users/3305a5ea28fe49369a8afdd55950ac18/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:50.750484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.779082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e85a2c-5c0c-4368-b61a-43041f0aecc0 service swift] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.870198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.871319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.872706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.873572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.875092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.877225 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2eb215-cc4a-438a-93fb-67e8d0eabbe0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.902066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1354/2668] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:50.903056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:50.917396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d0147-49ea-442c-8e7b-a892bf31299a admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:50.933470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:50.934240 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.934662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.935066 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] PATH_INFO: `/v3/projects/671c713c650c4b4cb562648fa367af37/users/3305a5ea28fe49369a8afdd55950ac18/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:50.944211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671c713c650c4b4cb562648fa367af37, user_id=3305a5ea28fe49369a8afdd55950ac18, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:50.947539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:50.981514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1315/2669] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:50 2022] PUT /identity/v3/projects/671c713c650c4b4cb562648fa367af37/users/3305a5ea28fe49369a8afdd55950ac18/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:58:50.994218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:50.994701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:50.995047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:51.102915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] MFA Rules not processed for user `3305a5ea28fe49369a8afdd55950ac18`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:51.104247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:51.286638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1355/2670] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:51.299083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:51.301504 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:51.377386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:51.385263 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b81f4d-4381-4156-973a-26d58976cba3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:51.397518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:51.398474 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:51.399398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:51.399837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:51.415579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:51.417936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48db7cd-d5e9-49a0-b2d0-97b27b1c60c0 service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:51.446268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1316/2671] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:52.103581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:52.201573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b049e72-afd5-4497-8f2e-4d341d17aab1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:52.207869 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:52.208696 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:52.208993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:52.209381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] PATH_INFO: `/v3/users/3948fbe798554fa8a82634f91eb6edca` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:52.210584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3948fbe798554fa8a82634f91eb6edca)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:52.216914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:52.230190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=100 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:52.236466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8916-6289-4960-8f12-502c0b1f02b2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:52.237031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8916-6289-4960-8f12-502c0b1f02b2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:52.237031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad8916-6289-4960-8f12-502c0b1f02b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:52.272221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-acad8916-6289-4960-8f12-502c0b1f02b2 None None] MFA Rules not processed for user `946a0fd33f754ce1aa534ab88dc95aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:52.274286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-acad8916-6289-4960-8f12-502c0b1f02b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:52.300437 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3948fbe798554fa8a82634f91eb6edca'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:52.315579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3948fbe798554fa8a82634f91eb6edca'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:52.357027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1317/2672] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:52.373908 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3948fbe798554fa8a82634f91eb6edca'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:52.386212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3948fbe798554fa8a82634f91eb6edca'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:52.387455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3948fbe798554fa8a82634f91eb6edca'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:52.396345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1356/2673] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:52 2022] DELETE /identity/v3/users/3948fbe798554fa8a82634f91eb6edca => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:52.401617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:52.401908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:52.402194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:52.425236 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:52.426612 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:52.532590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1318/2674] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:52.542976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:52.544424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:52.621515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90497d3-bd82-4316-b592-431db0d2ea28 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:52.625643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:52.626262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:52.626550 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:52.626825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:52.646351 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:52.647811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:52.676485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1357/2675] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:53.452836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:53.454949 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:53.479278 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d98e1a-714b-4c7c-95cc-0728790ffe99 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:53.485109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:53.485749 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:53.486033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:53.486307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] PATH_INFO: `/v3/projects/580c8efe91cc444aaf1fe0a1ab724c20` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:53.487450 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=580c8efe91cc444aaf1fe0a1ab724c20)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:53.489465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:53.552527 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580c8efe91cc444aaf1fe0a1ab724c20', 'request_id': 'req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.560170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580c8efe91cc444aaf1fe0a1ab724c20', 'request_id': 'req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.579139 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580c8efe91cc444aaf1fe0a1ab724c20', 'request_id': 'req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.597161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] The token cache is being invalidate because project 580c8efe91cc444aaf1fe0a1ab724c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:53.597455 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.598802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.604350 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1319/2676] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:53 2022] DELETE /identity/v3/projects/580c8efe91cc444aaf1fe0a1ab724c20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:53.605592 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:53.626528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2afe6-9be7-4f42-bc02-0393daab0d14 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:53.764209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:53.765194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:53.765472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:53.765803 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] PATH_INFO: `/v3/users/3305a5ea28fe49369a8afdd55950ac18` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:53.767069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3305a5ea28fe49369a8afdd55950ac18)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:53.769186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:53.826477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:53.853001 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3305a5ea28fe49369a8afdd55950ac18'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.861664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3305a5ea28fe49369a8afdd55950ac18'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.876927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3305a5ea28fe49369a8afdd55950ac18'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.895358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3305a5ea28fe49369a8afdd55950ac18'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.896466 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-4b8b96a6-3d65-404b-8ec3-baab1f5f29cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3305a5ea28fe49369a8afdd55950ac18'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:53.906072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1358/2677] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:53 2022] DELETE /identity/v3/users/3305a5ea28fe49369a8afdd55950ac18 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:54.266473 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.268012 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:54.299219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfedd2-13ec-4314-b41e-1d8eb8a4f458 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.324447 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:54.395224 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.395952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.396320 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.396706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] PATH_INFO: `/v3/projects/671c713c650c4b4cb562648fa367af37` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.397925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=671c713c650c4b4cb562648fa367af37)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.400207 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.459115 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671c713c650c4b4cb562648fa367af37', 'request_id': 'req-a844753b-db4a-415a-96de-fe53c5fbba1f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:54.465905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671c713c650c4b4cb562648fa367af37', 'request_id': 'req-a844753b-db4a-415a-96de-fe53c5fbba1f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:54.475085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671c713c650c4b4cb562648fa367af37', 'request_id': 'req-a844753b-db4a-415a-96de-fe53c5fbba1f'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:54.492511 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] The token cache is being invalidate because project 671c713c650c4b4cb562648fa367af37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:54.493336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:54.493989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a844753b-db4a-415a-96de-fe53c5fbba1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:54.495487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1320/2678] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:54 2022] DELETE /identity/v3/projects/671c713c650c4b4cb562648fa367af37 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:54.506341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.506842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.507220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.530425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:54.531839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:54.600780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1359/2679] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:54.611265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.614680 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.615107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.648110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:54.649841 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:54.660739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1321/2680] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:54.662981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.676942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b1f1-915b-452e-90b9-77cca12b7c3b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.686852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.690200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.690525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.690809 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.691626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:54.691927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.696849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.713496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1360/2681] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:54.717769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.730475 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f386df0-d5e9-46cd-ac26-b6d0b796bff2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.740077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.740668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.741016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.741414 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.742127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.744433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.796167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1322/2682] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:54 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:54.797416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.811500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461b9ed-e7bf-4a0e-990f-1202ae1ff0ad admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.879713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.880345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.880712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.880923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.881560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.883513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.908951 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1361/2683] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:54 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:54.912728 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.928368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2006d6-8c8a-4132-9752-4c08fe2fe775 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.933523 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.934140 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.934415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.934683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.935357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.937011 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.954292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1323/2684] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:54.955508 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:54.966724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b3d82-f61f-46c4-b7f0-10a91c981fa5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:54.974301 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:54.975204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:54.975564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:54.975965 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:54.977294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:54.979031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84895e64-1502-4269-8112-c679f7375c5e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:54.991359 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1362/2685] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:54.995735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.006992 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f1fb9-b1e1-4921-91c0-6cfe604cf4be admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.011822 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.015370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.015751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.016097 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] PATH_INFO: `/v3/projects/e3616d9a4f9048eeac8bfbbd719552f5/users/6eec7d5d9c8b4181b2eec8caa6621825/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.032629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3616d9a4f9048eeac8bfbbd719552f5, user_id=6eec7d5d9c8b4181b2eec8caa6621825, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.035378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.053476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1324/2686] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:54 2022] PUT /identity/v3/projects/e3616d9a4f9048eeac8bfbbd719552f5/users/6eec7d5d9c8b4181b2eec8caa6621825/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:55.068818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.069193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.069541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.105381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:55.110784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] MFA Rules not processed for user `6eec7d5d9c8b4181b2eec8caa6621825`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:55.112918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:55.234178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1363/2687] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:55.238802 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.367871 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:55.399863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a0493-a62c-4038-a54d-ca3768ade6ef admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.411361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.414868 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.415247 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.415469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.416221 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.419134 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.481641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1325/2688] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:55.482835 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.504448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330888cd-10b9-4109-ad4a-0f0d9168a171 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.614244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.616643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.616643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.616643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.616643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.619267 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.655846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1364/2689] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:55 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:55.657514 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.669031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfb293-6787-4512-89ab-8f5191f1dd62 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.674502 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.675378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.675661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.675937 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.676585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.678238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.697110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1326/2690] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:55.698526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.715626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b12a8-dc36-4e88-b186-37ef9763724b admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.722742 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.725496 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.728983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.729271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] PATH_INFO: `/v3/projects/ab2794a21ffd40cba4b4bb440890c7be/users/ab3d309bb615418e852d185ac6e562dc/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.750797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2794a21ffd40cba4b4bb440890c7be, user_id=ab3d309bb615418e852d185ac6e562dc, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.757902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.779772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1365/2691] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:55 2022] PUT /identity/v3/projects/ab2794a21ffd40cba4b4bb440890c7be/users/ab3d309bb615418e852d185ac6e562dc/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:55.781472 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.792676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9959c5d-d397-431f-9ff1-fd0c992d1252 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.868585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.868847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.869109 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.869392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.870031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.871707 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092bc7e6-501d-4fb9-b567-d639e50f6761 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.890660 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1327/2692] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:55.894552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:55.906288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a27066-acf7-47a5-b674-98a79d565e14 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:55.921492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:55.922259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.922541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.922817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] PATH_INFO: `/v3/projects/ab2794a21ffd40cba4b4bb440890c7be/users/ab3d309bb615418e852d185ac6e562dc/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.925053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2794a21ffd40cba4b4bb440890c7be, user_id=ab3d309bb615418e852d185ac6e562dc, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:55.928111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:55.950724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1366/2693] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:55 2022] PUT /identity/v3/projects/ab2794a21ffd40cba4b4bb440890c7be/users/ab3d309bb615418e852d185ac6e562dc/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:55.952676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:55.953040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:55.953374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:55.996113 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] MFA Rules not processed for user `ab3d309bb615418e852d185ac6e562dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:55.997797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:56.126981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1328/2694] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:56.146725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:56.157549 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:56.243320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f802e03-b67f-4e5c-bcd2-bcc44c64a84f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:56.247598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:56.248436 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:56.249011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:56.249438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:56.262387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:56.264285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:56.269762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:56.291259 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1367/2695] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:58:56.411250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:56.436544 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b32c8-5e08-4fc8-b1bd-76107cbed4b2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:56.443621 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:56.444481 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:56.444809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:56.445069 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:56.555960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:56.602734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:56.604202 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf6f2d-85a6-4966-bb2c-d6cdeb1441eb service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:56.623234 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1329/2696] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:58:57.065687 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:57.166445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3fd68-7f43-4fc6-a860-be478c2b955c service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:57.171115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:57.171877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:57.172269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:57.172593 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] PATH_INFO: `/v3/users/6eec7d5d9c8b4181b2eec8caa6621825` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:57.173979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eec7d5d9c8b4181b2eec8caa6621825)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:57.176278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:57.247659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eec7d5d9c8b4181b2eec8caa6621825'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:57.256663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eec7d5d9c8b4181b2eec8caa6621825'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:57.275955 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eec7d5d9c8b4181b2eec8caa6621825'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:57.288183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eec7d5d9c8b4181b2eec8caa6621825'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:57.289646 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eec7d5d9c8b4181b2eec8caa6621825'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:57.302015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1368/2697] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:57 2022] DELETE /identity/v3/users/6eec7d5d9c8b4181b2eec8caa6621825 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:57.303426 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:57.303714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:57.304104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:57.327788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:57.329340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:57.434955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1330/2698] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:58:57.448747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:57.450411 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:57.521992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15164295-ea7a-4e7f-b7f5-335e80f3b9f0 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:57.526539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:57.527312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:57.527595 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:57.527865 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:57.538262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:57.539848 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:57.552068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1369/2699] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:58:58.031694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:58.033624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:58.055067 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691467d-79d5-4c91-beeb-41feecfc61e7 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:58.061955 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:58.062805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:58.063147 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:58.063464 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] PATH_INFO: `/v3/projects/e3616d9a4f9048eeac8bfbbd719552f5` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:58.064676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3616d9a4f9048eeac8bfbbd719552f5)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:58.070910 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:58.170274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3616d9a4f9048eeac8bfbbd719552f5', 'request_id': 'req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.178083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3616d9a4f9048eeac8bfbbd719552f5', 'request_id': 'req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.189216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3616d9a4f9048eeac8bfbbd719552f5', 'request_id': 'req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.208739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] The token cache is being invalidate because project e3616d9a4f9048eeac8bfbbd719552f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:58.208739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.215780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.219399 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1331/2700] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:58 2022] DELETE /identity/v3/projects/e3616d9a4f9048eeac8bfbbd719552f5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:58.228915 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:58.252010 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8e995-61a5-4fb3-9566-e02132028c68 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:58.304258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:58.389165 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:58.390057 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:58.390388 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:58.390664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] PATH_INFO: `/v3/users/ab3d309bb615418e852d185ac6e562dc` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:58.391794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3d309bb615418e852d185ac6e562dc)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:58.393753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:58.450339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3d309bb615418e852d185ac6e562dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.463196 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3d309bb615418e852d185ac6e562dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.479586 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3d309bb615418e852d185ac6e562dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.490645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3d309bb615418e852d185ac6e562dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.491572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-a24d75f0-bc7e-4800-83e2-25ec61144139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3d309bb615418e852d185ac6e562dc'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:58.499446 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1370/2701] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:58:58 2022] DELETE /identity/v3/users/ab3d309bb615418e852d185ac6e562dc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:58.884539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:58.886153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:58.922094 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a419ed7-24ba-4ce8-9f25-bc3197b094aa admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:58.998936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:58.999699 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.000042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.000417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] PATH_INFO: `/v3/projects/ab2794a21ffd40cba4b4bb440890c7be` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.001624 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2794a21ffd40cba4b4bb440890c7be)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.003594 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.006441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:59.066380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2794a21ffd40cba4b4bb440890c7be', 'request_id': 'req-09eea231-f6a1-4326-a3f8-634b1690e729'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:59.075028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2794a21ffd40cba4b4bb440890c7be', 'request_id': 'req-09eea231-f6a1-4326-a3f8-634b1690e729'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:59.085433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2794a21ffd40cba4b4bb440890c7be', 'request_id': 'req-09eea231-f6a1-4326-a3f8-634b1690e729'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:59.114729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] The token cache is being invalidate because project ab2794a21ffd40cba4b4bb440890c7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:58:59.114729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:59.114729 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-09eea231-f6a1-4326-a3f8-634b1690e729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:58:59.126265 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1332/2702] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:58:58 2022] DELETE /identity/v3/projects/ab2794a21ffd40cba4b4bb440890c7be => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:58:59.134537 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.134900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.135254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.169139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:59.172884 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:59.246132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1371/2703] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:59.316122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.318942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.318942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.339973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:59.341286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:59.349837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1333/2704] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:59.353817 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.364264 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439364d5-ca9d-4a53-9c15-e9225de25b3e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.375345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.376033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.376767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.376767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.377567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:58:59.377926 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.379554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.396942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1372/2705] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:58:59.401348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.413458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a5c1e-9225-42cb-b3fe-ce606a0aca58 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.418944 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.419737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.420205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.420681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.421531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.423602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.469282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1334/2706] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:59.474146 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.484509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00893ead-a34c-4b91-b330-e9f13b968037 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.517348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:58:59.564679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.565383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.565663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.565935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.566693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.568647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.598801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1373/2707] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:58:59.603398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.614613 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf193-e83c-4046-8840-fdac6bbef9a0 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.620465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.621111 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.621393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.621661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.622617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.628817 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.636741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1335/2708] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:59.641430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.652493 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2726-bfab-44b2-883e-ca7e58b3f960 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.661161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.661771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.662068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.662527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.663093 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.665395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076deeee-4364-4b60-ab9a-bc9897b7cc54 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.678793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1374/2709] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:58:59.684836 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.696150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b88b8-2fde-47ae-b6d6-4765399ff3ce admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.703818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.704626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.705005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.705347 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] PATH_INFO: `/v3/projects/46bd8e25f82545f397e7bc3b79bbc2e2/users/195f67920ff34490b1cabba080421d77/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.722622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=46bd8e25f82545f397e7bc3b79bbc2e2, user_id=195f67920ff34490b1cabba080421d77, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.725304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:58:59.743911 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1336/2710] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:58:59 2022] PUT /identity/v3/projects/46bd8e25f82545f397e7bc3b79bbc2e2/users/195f67920ff34490b1cabba080421d77/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:58:59.754992 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.755268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.755543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.789296 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] MFA Rules not processed for user `195f67920ff34490b1cabba080421d77`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:58:59.790856 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:58:59.874674 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1375/2711] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:58:59.879171 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:58:59.959568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6de1d4-a81c-46aa-bcbe-0d07bb2515ad admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:58:59.966108 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:58:59.966974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:58:59.967264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:58:59.967540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:58:59.968188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:58:59.970170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.006264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1337/2712] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:58:59 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 04:59:00.011154 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.031536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e716-6b15-418a-9d7a-b2890d06f5d9 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.101282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.102556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.102841 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.103244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.103894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.105830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.142302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1376/2713] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:59:00 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:59:00.143829 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.159490 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76eae0-6da4-4212-9284-e9200102bdb3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.164890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.165576 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.165853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.166131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.166801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.168732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.183351 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1338/2714] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:59:00.187665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.198931 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0a7b9-0fa0-48cb-8227-1c2b42fa551a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.203753 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.204425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.204718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.205045 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] PATH_INFO: `/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383/users/b0890c2569104e50a73ba298c54098a1/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.222058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce43fc3ae1a041ab829bdcca7e3af383, user_id=b0890c2569104e50a73ba298c54098a1, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.225979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.238512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:00.252707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1377/2715] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:59:00 2022] PUT /identity/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383/users/b0890c2569104e50a73ba298c54098a1/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:00.254791 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.262104 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:00.273413 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83efd0f-546c-48f2-a4d8-0bdd6f5cd2d2 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.372636 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.373411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.373794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.374326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.375079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.376839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75921557-9050-453a-b1a7-92d9de8c6c63 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.393245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1339/2716] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 04:59:00.397274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.409785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d467d97-4313-4e22-8694-6b74caea26cd admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.415786 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.416548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.417000 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.417348 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] PATH_INFO: `/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383/users/b0890c2569104e50a73ba298c54098a1/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.419608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce43fc3ae1a041ab829bdcca7e3af383, user_id=b0890c2569104e50a73ba298c54098a1, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.422756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.438377 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1378/2717] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:59:00 2022] PUT /identity/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383/users/b0890c2569104e50a73ba298c54098a1/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:59:00.448801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.449235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.449908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.487897 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] MFA Rules not processed for user `b0890c2569104e50a73ba298c54098a1`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:00.489429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:00.574123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1340/2718] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:00.585038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:00.587080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:00.668995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200beb7-65ac-4966-b326-2185a764438f admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:00.673212 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:00.674030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:00.674299 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:00.674567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:00.753718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:00.755169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268687c-0ffe-4acd-97b8-304a84ea69f4 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:00.767612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1379/2719] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:05.110770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:05.112241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:05.124586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cfa74-49f5-474c-a9ec-dba910244dce None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:05.128328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:05.129363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:05.129650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:05.129924 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:05.139643 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:05.141148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8430d37-e940-4aa0-befa-701b810d991f service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:05.155052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1341/2720] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:07.430974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.431570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.431570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.433128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1380/2721] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:59:07.439182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d442e-4d3e-43c7-b717-78d3b428c741 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.439623 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d442e-4d3e-43c7-b717-78d3b428c741 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.439842 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d442e-4d3e-43c7-b717-78d3b428c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.460200 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-553d442e-4d3e-43c7-b717-78d3b428c741 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:07.462090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-553d442e-4d3e-43c7-b717-78d3b428c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.530763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1342/2722] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:07.541657 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:07.543363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.602723 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0361-6285-4c9e-952e-f8c29051f528 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:07.606416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:07.607009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.607284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.607554 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.618078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:07.619426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851b204-55eb-44fa-b62a-c6d7f9a02640 service glance] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:07.629935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1381/2723] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:59:07.699899 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.700188 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.700496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.701962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1343/2724] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:59:07.706807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e438e9e-6708-4581-adab-ea67b9f49812 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.707089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e438e9e-6708-4581-adab-ea67b9f49812 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.707356 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e438e9e-6708-4581-adab-ea67b9f49812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.726767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-2e438e9e-6708-4581-adab-ea67b9f49812 None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:07.727957 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2e438e9e-6708-4581-adab-ea67b9f49812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.767536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:07.779627 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1382/2725] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:07.791790 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:07.793498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.801152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:07.808741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b4f57-b50a-4307-b170-63b247cd4af8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:07.812258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41873-f234-4721-9153-886778986c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:07.812886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41873-f234-4721-9153-886778986c9b service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.813166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41873-f234-4721-9153-886778986c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.813436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41873-f234-4721-9153-886778986c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.820990 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea41873-f234-4721-9153-886778986c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:07.822417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea41873-f234-4721-9153-886778986c9b service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:07.830384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1344/2726] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:07.921538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.921921 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.922288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.923619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1383/2727] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 04:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 04:59:07.931669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.931960 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.932230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:07.953491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:07.954673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.964078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1345/2728] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:59:07.974419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:07.976725 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:07.990410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb4806-344c-4ce5-b369-9e1bdb4c7626 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:07.994801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:07.995381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:07.995654 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:07.995919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:08.005760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:08.007186 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2e6d9-5470-4884-99d9-4a40368d016a service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:08.017953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1384/2729] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:10.111226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:10.179670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be36a-e8e9-469f-8e06-cd0bd4d05650 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:10.184090 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:10.184752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:10.185038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:10.185311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] PATH_INFO: `/v3/users/195f67920ff34490b1cabba080421d77` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:10.186460 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=195f67920ff34490b1cabba080421d77)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:10.188392 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:10.237931 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '195f67920ff34490b1cabba080421d77'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.245142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '195f67920ff34490b1cabba080421d77'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.257648 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '195f67920ff34490b1cabba080421d77'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.266463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '195f67920ff34490b1cabba080421d77'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.267073 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '195f67920ff34490b1cabba080421d77'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.273622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1346/2730] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:59:10 2022] DELETE /identity/v3/users/195f67920ff34490b1cabba080421d77 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:10.278560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:10.279381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:10.279946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:10.299247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:10.300470 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:10.362161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1385/2731] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 04:59:10.371801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:10.373600 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:10.435908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237648a-ac5c-4f63-b69b-2f6ed214f266 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:10.439437 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:10.440042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:10.440319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:10.440585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:10.450258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:10.451693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:10.461863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1347/2732] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:10.749156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:10.750766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:10.764859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a71dfe-ffaf-415c-bf9e-630b855061dd None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:10.773855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:10.774504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:10.774785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:10.775058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] PATH_INFO: `/v3/projects/46bd8e25f82545f397e7bc3b79bbc2e2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:10.776178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46bd8e25f82545f397e7bc3b79bbc2e2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:10.778573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:10.828799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46bd8e25f82545f397e7bc3b79bbc2e2', 'request_id': 'req-d383991f-487b-4982-a2e5-53ffc5f52404'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.834287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46bd8e25f82545f397e7bc3b79bbc2e2', 'request_id': 'req-d383991f-487b-4982-a2e5-53ffc5f52404'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.842943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46bd8e25f82545f397e7bc3b79bbc2e2', 'request_id': 'req-d383991f-487b-4982-a2e5-53ffc5f52404'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.856907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] The token cache is being invalidate because project 46bd8e25f82545f397e7bc3b79bbc2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:59:10.857192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.857806 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d383991f-487b-4982-a2e5-53ffc5f52404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:10.859291 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1386/2733] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:59:10 2022] DELETE /identity/v3/projects/46bd8e25f82545f397e7bc3b79bbc2e2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:59:10.863319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:10.869786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:10.873929 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bf8a6-b838-4d86-8621-6be25ff6f9d0 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:10.938499 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:10.938732 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:10.938945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:10.939211 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] PATH_INFO: `/v3/users/b0890c2569104e50a73ba298c54098a1` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:10.940322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0890c2569104e50a73ba298c54098a1)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:10.942398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:10.994907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0890c2569104e50a73ba298c54098a1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:11.003440 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0890c2569104e50a73ba298c54098a1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:11.017389 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0890c2569104e50a73ba298c54098a1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:11.027483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0890c2569104e50a73ba298c54098a1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:11.028999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0890c2569104e50a73ba298c54098a1'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:11.040573 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1348/2734] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:59:10 2022] DELETE /identity/v3/users/b0890c2569104e50a73ba298c54098a1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:11.512626 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:11.513157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:11.513321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:11.533854 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:11.535192 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:11.592624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1387/2735] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 04:59:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:11.605303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:11.606760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:11.676798 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bbc54-797c-42da-94b2-074aeabe3abc admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:11.683159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:11.683770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:11.684048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:11.684315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:11.692655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:11.694235 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f24760-da9e-43c8-953b-b8bebe546b87 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:11.702322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1349/2736] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:59:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:59:11.847029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:11.849408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:11.872969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:12.023363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140d1ae-6c44-4ec8-b9b3-c74b0a97f512 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.032275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.034229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.034794 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.035277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] PATH_INFO: `/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.037181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce43fc3ae1a041ab829bdcca7e3af383)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.040236 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.121939 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce43fc3ae1a041ab829bdcca7e3af383', 'request_id': 'req-298a25a8-bad5-477d-94fc-5d05cc497778'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:12.132526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce43fc3ae1a041ab829bdcca7e3af383', 'request_id': 'req-298a25a8-bad5-477d-94fc-5d05cc497778'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:12.142979 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce43fc3ae1a041ab829bdcca7e3af383', 'request_id': 'req-298a25a8-bad5-477d-94fc-5d05cc497778'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:12.168094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] The token cache is being invalidate because project ce43fc3ae1a041ab829bdcca7e3af383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:59:12.168465 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:12.169426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-298a25a8-bad5-477d-94fc-5d05cc497778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:12.171561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1388/2737] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:59:11 2022] DELETE /identity/v3/projects/ce43fc3ae1a041ab829bdcca7e3af383 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:12.177943 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.178222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.178569 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.239322 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:12.242128 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:12.271849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:12.367064 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1350/2738] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:12.369189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.369475 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.369876 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.416760 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:12.418474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:12.440880 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1389/2739] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:12.445304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.458258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca866fe4-7b60-41e1-bfcf-303bde6d31e1 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.477267 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.487479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.488204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.491847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.492884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 04:59:12.493403 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.499116 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.522801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1351/2740] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 04:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 04:59:12.523833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.542216 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858036b-b4f8-41b7-b121-f4186feb74a8 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.553656 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.554320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.554597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.554868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.555512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.557478 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.647452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1390/2741] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 04:59:12 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:59:12.648743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.660141 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8d91-cb6f-41d6-904c-a400061e333e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.729117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.729805 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.730150 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.730483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.731185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.733411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.762050 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1352/2742] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 04:59:12 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 04:59:12.766405 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.779597 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a6585-d2aa-4e6e-a2f5-89f2063456fe admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.791464 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.792284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.796686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.797120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.797877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.799561 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.815793 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1391/2743] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:59:12.819849 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.830743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9d522-4993-4f48-93a3-9b6f663899c4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.836251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.836886 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.837164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.837436 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.838076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.839702 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbe15e-b57a-439b-82aa-8b85939948e9 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.856233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1353/2744] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 04:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 04:59:12.857448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:12.869032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae58b-2106-44f9-8897-c97006fa2055 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:12.875679 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:12.876355 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.876670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.876934 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] PATH_INFO: `/v3/projects/f5fecf683f6942508a6b2897be4418a7/users/7f4ba8108dd44e7186e928f9501205bb/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.893735 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fecf683f6942508a6b2897be4418a7, user_id=7f4ba8108dd44e7186e928f9501205bb, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:12.897294 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:12.914994 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1392/2745] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 04:59:12 2022] PUT /identity/v3/projects/f5fecf683f6942508a6b2897be4418a7/users/7f4ba8108dd44e7186e928f9501205bb/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 04:59:12.926536 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:12.926903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:12.927148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:12.958854 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] MFA Rules not processed for user `7f4ba8108dd44e7186e928f9501205bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:12.961838 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:13.060384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1354/2746] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:13.068381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:13.072469 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:13.118452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:13.164704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1f54f-e6f7-49ad-b242-5258e1c80304 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:13.169532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:13.170419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:13.175062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:13.175424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:13.187528 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:13.189163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e2301-a3ce-4403-ae3e-a9b6166fb84e service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:13.202819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1393/2747] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 04:59:13.230742 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:13.242213 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:13.312762 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bbaa0-3c44-4c85-aacd-61c040b1bc82 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:13.317945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:13.318599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:13.318878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:13.319152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] PATH_INFO: `/v3/users/7f4ba8108dd44e7186e928f9501205bb` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:13.320338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4ba8108dd44e7186e928f9501205bb)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:13.322311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:13.381776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4ba8108dd44e7186e928f9501205bb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:13.390908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4ba8108dd44e7186e928f9501205bb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:13.414286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4ba8108dd44e7186e928f9501205bb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:13.423277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4ba8108dd44e7186e928f9501205bb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:13.423894 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4ba8108dd44e7186e928f9501205bb'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:13.431361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1355/2748] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 04:59:13 2022] DELETE /identity/v3/users/7f4ba8108dd44e7186e928f9501205bb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:13.435677 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:13.435959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:13.436234 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:13.457662 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:13.459522 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:13.535717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1394/2749] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 04:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:13.546388 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:13.547888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:13.637796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968af905-8990-4383-b7aa-dc9d1e39351f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:13.642229 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:13.643467 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:13.644164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:13.644950 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:13.660735 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:13.662428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dc1ab-ab87-4818-894e-abd2206831d7 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:13.676100 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1356/2750] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 04:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 04:59:14.086943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:14.088134 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:14.101497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a900-d44f-4fc4-8f46-342bccb3fbbb None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:14.106514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:14.107204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:14.107542 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:14.107745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] PATH_INFO: `/v3/projects/f5fecf683f6942508a6b2897be4418a7` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:14.109370 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5fecf683f6942508a6b2897be4418a7)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:14.111536 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:14.202230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5fecf683f6942508a6b2897be4418a7', 'request_id': 'req-c4e6c8e7-5175-413c-ac4b-fa77107357b4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:14.214548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5fecf683f6942508a6b2897be4418a7', 'request_id': 'req-c4e6c8e7-5175-413c-ac4b-fa77107357b4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:14.222867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5fecf683f6942508a6b2897be4418a7', 'request_id': 'req-c4e6c8e7-5175-413c-ac4b-fa77107357b4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:14.254140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] The token cache is being invalidate because project f5fecf683f6942508a6b2897be4418a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 04:59:14.254949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:14.255673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 04:59:14.258072 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1395/2751] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 04:59:14 2022] DELETE /identity/v3/projects/f5fecf683f6942508a6b2897be4418a7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 04:59:23.670027 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:24.250839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:32.334433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc4c0-32a1-4a7a-8998-8929c939f8ed None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:32.334918 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc4c0-32a1-4a7a-8998-8929c939f8ed None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:32.335016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc4c0-32a1-4a7a-8998-8929c939f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:32.356692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3b5fc4c0-32a1-4a7a-8998-8929c939f8ed None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 04:59:32.357936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3b5fc4c0-32a1-4a7a-8998-8929c939f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:32.413785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1357/2752] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 04:59:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 04:59:32.426959 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 04:59:32.428400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 04:59:32.496798 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c8e7-5175-413c-ac4b-fa77107357b4 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 04:59:32.501771 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 04:59:32.502369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 04:59:32.502651 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 04:59:32.502923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 04:59:32.511449 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 04:59:32.513064 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 04:59:32.521369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1396/2753] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 04:59:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 04:59:42.407005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=25 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 04:59:42.517969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=32 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:00:13.405768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00a302-6cbf-4d46-b175-a2dc9214084c None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:00:13.406328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00a302-6cbf-4d46-b175-a2dc9214084c None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:00:13.406328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00a302-6cbf-4d46-b175-a2dc9214084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:00:13.434563 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-ec00a302-6cbf-4d46-b175-a2dc9214084c None None] MFA Rules not processed for user `32cbf88b156745ae9a59e8f795e8ef58`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:00:13.435757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ec00a302-6cbf-4d46-b175-a2dc9214084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:00:13.503423 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1358/2754] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:00:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:00:14.562927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:00:14.564323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:00:14.690526 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ca54e-000f-48c8-95a5-6ef5497e4eb2 service swift] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:00:14.695456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:00:14.696046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:00:14.696320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:00:14.696587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:00:14.698289 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:00:14.709254 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:00:14.710837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:00:14.720497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1397/2755] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:00:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:00:23.501667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=37 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:00:24.716300 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=59 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:00:46.973380 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826ce9-8367-4a70-8db1-6b22f6a9413b None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:00:46.973956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826ce9-8367-4a70-8db1-6b22f6a9413b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:00:46.973956 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4826ce9-8367-4a70-8db1-6b22f6a9413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:00:46.993107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f4826ce9-8367-4a70-8db1-6b22f6a9413b None None] MFA Rules not processed for user `946a0fd33f754ce1aa534ab88dc95aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:00:46.994302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f4826ce9-8367-4a70-8db1-6b22f6a9413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:00:47.048158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1359/2756] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:00:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:00:57.048196 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=25 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:01.730258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:01.731664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:01.830783 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f363707-c6ca-4246-8570-c46c7c06357f service placement] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:01.836896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:01.837494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:01.837774 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:01.838046 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:01.839715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:01.849898 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:01.851461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:01.860249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1398/2757] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:01:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:01:11.856145 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=46 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:21.270804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990a039-50e4-41fd-8b77-7f8fb7325376 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:21.271335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990a039-50e4-41fd-8b77-7f8fb7325376 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:21.271449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990a039-50e4-41fd-8b77-7f8fb7325376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:21.303510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-f990a039-50e4-41fd-8b77-7f8fb7325376 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:21.304780 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f990a039-50e4-41fd-8b77-7f8fb7325376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:21.312857 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1360/2758] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:01:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:21.362316 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:21.374912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9946277c-582a-452a-a0a0-2a6c068f1366 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:21.379896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:21.380486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:21.380796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:21.381068 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:21.389218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:21.390801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1d255-29a1-43c6-b72a-6cc3be8d3e45 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:21.398830 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1399/2759] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:01:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:01:31.297945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=7 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:31.395733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=6 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:34.915476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:34.916037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:34.916037 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:34.917394 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1361/2760] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 05:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 05:01:34.923515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d5ae8-2f9b-4a42-ba9a-91638aeb1680 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:34.924012 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d5ae8-2f9b-4a42-ba9a-91638aeb1680 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:34.924126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d5ae8-2f9b-4a42-ba9a-91638aeb1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:34.956527 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-da6d5ae8-2f9b-4a42-ba9a-91638aeb1680 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:34.957778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-da6d5ae8-2f9b-4a42-ba9a-91638aeb1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:34.968071 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1400/2761] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 05:01:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:34.980292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:34.982051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:35.045500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a9b53-1b58-4302-b63c-33cdea284f75 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:35.049042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:35.049652 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.049933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.050206 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.060947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:35.062368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adaea9-83f3-4eee-875a-9ac0fac1fe87 service glance] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:35.072891 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1362/2762] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:01:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:01:35.142229 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.142515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.142787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] PATH_INFO: `/` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.144116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1401/2763] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 05:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 05:01:35.150052 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed8e0-88cb-4348-b1ed-63b4b4bf5319 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.150348 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed8e0-88cb-4348-b1ed-63b4b4bf5319 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.150629 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ed8e0-88cb-4348-b1ed-63b4b4bf5319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.182142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9f9ed8e0-88cb-4348-b1ed-63b4b4bf5319 None None] MFA Rules not processed for user `2fe2203ea1f94623956a215190e4d79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:35.183335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9f9ed8e0-88cb-4348-b1ed-63b4b4bf5319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:35.242572 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1363/2764] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 05:01:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:35.254403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:35.256193 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:35.269973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f39743-d933-4c9f-b3a4-e750be7581f1 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:35.273456 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:35.274036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.274313 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.274585 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.281434 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:35.282840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992cfa55-f5aa-4166-9181-6a10ecd6cfd3 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:35.291050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1402/2765] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:01:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:01:35.374261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.374550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.374828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] PATH_INFO: `/` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.376189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1364/2766] 10.222.0.27 () {54 vars in 921 bytes} [Tue Sep 20 05:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 20 05:01:35.381941 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ec20-a3b4-4549-a7c9-5e458895fbe2 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.382227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ec20-a3b4-4549-a7c9-5e458895fbe2 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.382504 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6ec20-a3b4-4549-a7c9-5e458895fbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.403546 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-5cf6ec20-a3b4-4549-a7c9-5e458895fbe2 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:35.404779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5cf6ec20-a3b4-4549-a7c9-5e458895fbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:35.414659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1403/2767] 10.222.0.27 () {58 vars in 1018 bytes} [Tue Sep 20 05:01:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:35.425382 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:35.427071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:35.440412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80540175-4b63-4e76-b045-9aa29539adbc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:35.444531 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:35.445148 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:35.445429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:35.445697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:35.455641 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:35.457721 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:35.468028 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1365/2768] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:01:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:01:37.246021 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19fa2-b06b-49c8-9752-ae87fa6866af None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:37.246512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19fa2-b06b-49c8-9752-ae87fa6866af None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:37.246615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19fa2-b06b-49c8-9752-ae87fa6866af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:37.265840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4cc19fa2-b06b-49c8-9752-ae87fa6866af None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:37.267018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4cc19fa2-b06b-49c8-9752-ae87fa6866af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:37.274085 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1404/2769] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:37.286966 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:37.298622 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6fff0-ff7b-463f-ac78-309f88c09c80 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:37.304722 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:37.305344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:37.305627 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:37.305905 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:37.314601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:37.316197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:37.325428 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1366/2770] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:01:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:01:47.260109 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=19 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:47.322123 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=70 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:01:51.048288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:51.048770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:51.048907 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:51.069629 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:01:51.070837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:01:51.078107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1405/2771] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 05:01:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:01:51.090005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:01:51.100232 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4994-793d-4b68-af8e-e02fd686c9bf service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:01:51.105205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:01:51.105792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:01:51.106070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:01:51.106337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:01:51.114601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:01:51.116169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:01:51.124004 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1367/2772] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:01:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 05:02:01.063419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:01.120819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=6 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:25.681768 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:25.691078 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73c306-e3e3-4e04-93bb-3ac95a9ad097 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:25.694663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:25.695253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:25.695525 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:25.695787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:25.697530 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:25.711050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:25.712442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:25.723697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1406/2773] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:28.011169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:28.023999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee412a0-1815-4b43-a6d3-086c957d40c1 service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:28.030626 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:28.031245 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:28.031558 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:28.031896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:28.033664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:28.046964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:28.048694 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:28.064040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1368/2774] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:34.790946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:34.800176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3812e-9786-45bc-b2b1-fcd48145e7a3 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:34.804442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:34.805066 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:34.805343 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:34.805613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:34.807319 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:34.838241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:34.848537 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87f9d1-5eba-4166-a106-b62861ae1067 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:34.856647 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:34.859127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:34.859415 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:34.859692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:34.862970 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:34.875186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:34.876927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:34.888244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1369/2775] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:34.911168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:34.912752 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:34.934609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1407/2776] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:36.111936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.113620 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.130349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add461b-b24e-4101-b1a4-9d6084d2fcec service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.137010 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.137700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.138057 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.138396 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.140734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.156693 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.158312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:36.169915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1370/2777] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 05:02:36.255574 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.257062 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.270201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db533ba3-ab7a-4d30-a60a-7a26429af042 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.275168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.275935 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.276282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.276645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] PATH_INFO: `/v3/users/32cbf88b156745ae9a59e8f795e8ef58` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.277857 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=32cbf88b156745ae9a59e8f795e8ef58)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.280647 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:36.339402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.342274 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32cbf88b156745ae9a59e8f795e8ef58'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.350772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32cbf88b156745ae9a59e8f795e8ef58'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.354195 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2f67e-1875-47f0-a9a1-3d05c76fffb7 service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.361763 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.362363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.362874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.362971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.365287 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.365427 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32cbf88b156745ae9a59e8f795e8ef58'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.374074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32cbf88b156745ae9a59e8f795e8ef58'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.374737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32cbf88b156745ae9a59e8f795e8ef58'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.378422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.380754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcbc8d0-d6dc-4d92-892a-84a4d8f54878 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:36.382127 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1408/2778] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:02:36 2022] DELETE /identity/v3/users/32cbf88b156745ae9a59e8f795e8ef58 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:02:36.392276 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1371/2779] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:36.565609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.567663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.608762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:36.652571 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e154cd-979c-408f-8a38-cabfba175bbf admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.660253 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.661754 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.662298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.662670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] PATH_INFO: `/v3/projects/56047142d69c4c1cbfa9f6a278622332` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.664077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56047142d69c4c1cbfa9f6a278622332)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.666731 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:36.720849 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56047142d69c4c1cbfa9f6a278622332', 'request_id': 'req-40044827-c1b5-42ac-9128-7e48396b32b2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.727107 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56047142d69c4c1cbfa9f6a278622332', 'request_id': 'req-40044827-c1b5-42ac-9128-7e48396b32b2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.733661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56047142d69c4c1cbfa9f6a278622332', 'request_id': 'req-40044827-c1b5-42ac-9128-7e48396b32b2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.746953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] The token cache is being invalidate because project 56047142d69c4c1cbfa9f6a278622332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:02:36.747245 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.747866 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-40044827-c1b5-42ac-9128-7e48396b32b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:36.752846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1409/2780] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:02:36 2022] DELETE /identity/v3/projects/56047142d69c4c1cbfa9f6a278622332 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:02:36.754631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.754921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.755204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.815122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:02:36.817065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.878462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1372/2781] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:02:36.886412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.886721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.886977 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.906704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:02:36.907912 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:36.916430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1410/2782] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:02:36.920566 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.930124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd4f0c-1cba-4c2f-af6a-e3695bd2fbb5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.934664 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.935259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.935540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.935814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.936476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:02:36.936806 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.938402 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:36.949639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1373/2783] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:02:36.953673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:36.963257 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f3b09-d9d3-4ef1-815d-e5ba1bc2cf2a None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:36.967834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:36.968418 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:36.968729 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:36.969002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:36.969624 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:36.971507 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.006940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1411/2784] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:02:36 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:02:37.011078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:37.021668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210081df-81dc-4729-b278-c9012ee7d2ae admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:37.078303 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:37.079144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.079425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.079873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.080551 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:37.083006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.104977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1374/2785] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:02:37 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:02:37.109160 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:37.120974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3ec8c-9a9d-451a-aa0e-a1a0c89b14b6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:37.128581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:37.129619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.130091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.130564 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.131698 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:37.133882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.150751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1412/2786] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:02:37.152364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:37.158182 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:37.165434 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186143fd-4787-46c9-8d17-b3a81e52466e admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:37.172890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:37.173543 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.173889 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.174261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.175628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:37.177550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced69bc-6815-4c05-a045-7115cd4d8b19 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.188698 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1375/2787] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:02:37.192986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:37.204002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911c85f-8ed8-408c-b708-370c82d8e54c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:37.209357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:37.210033 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.210309 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.210580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] PATH_INFO: `/v3/projects/3218048072f048d9afc65ae138523bfd/users/1947f5ce1d794f05ae6272e8dba5d29c/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.225738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3218048072f048d9afc65ae138523bfd, user_id=1947f5ce1d794f05ae6272e8dba5d29c, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:37.228364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.241584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1413/2788] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:02:37 2022] PUT /identity/v3/projects/3218048072f048d9afc65ae138523bfd/users/1947f5ce1d794f05ae6272e8dba5d29c/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:02:37.246548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136bfb-f73d-42bb-a7bc-60ea1d147744 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.246834 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136bfb-f73d-42bb-a7bc-60ea1d147744 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.247107 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40136bfb-f73d-42bb-a7bc-60ea1d147744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.266642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-40136bfb-f73d-42bb-a7bc-60ea1d147744 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:02:37.267848 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-40136bfb-f73d-42bb-a7bc-60ea1d147744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:37.324688 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1376/2789] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:02:37.335030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:37.336852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:37.387512 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:37.401044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c95712-79a0-4005-a3cc-f10b02c7e67a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:37.405132 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:37.405864 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:37.406128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:37.406390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:37.415807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:37.417249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:37.428855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1414/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:41.733583 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8dfcb-bb48-4e57-bcb1-d395d015aead None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:41.734058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8dfcb-bb48-4e57-bcb1-d395d015aead None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:41.734058 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8dfcb-bb48-4e57-bcb1-d395d015aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:41.790684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-abe8dfcb-bb48-4e57-bcb1-d395d015aead None None] MFA Rules not processed for user `1947f5ce1d794f05ae6272e8dba5d29c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:02:41.791895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-abe8dfcb-bb48-4e57-bcb1-d395d015aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:41.872874 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1377/2791] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:02:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:02:41.886975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:41.888737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:41.983970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b483f-3dd9-4a5b-8fd1-01adcc25a518 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:41.991917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:41.992633 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:41.992913 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:41.993184 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:42.002699 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:42.004227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:42.014167 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1415/2792] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:44.085410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:44.085890 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:44.086085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:44.112975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] MFA Rules not processed for user `1947f5ce1d794f05ae6272e8dba5d29c`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:02:44.115157 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:44.126902 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1378/2793] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:02:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:02:44.470779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:44.472201 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:44.484716 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c45753-e5ae-4b37-9f11-00b30d9b00f2 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:44.488378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:44.489538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:44.489813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:44.490082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:44.491763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:44.502286 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:44.503932 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:44.513171 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1416/2794] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:44.856945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:44.858579 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:44.951589 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb27956-cd97-40d4-aeb4-1b063a5070c4 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:44.961168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:44.961909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:44.962427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:44.962704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:44.969170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:44.985300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:44.987259 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:45.002186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1379/2795] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:46.846025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:46.855734 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7e2e7-699d-4dc4-85db-cab85eece8cd service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:46.861183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:46.861942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:46.862277 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:46.862611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:46.864579 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:46.876625 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:46.878149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:46.896565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1417/2796] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:51.750143 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:51.759685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=100 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:51.766209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28747ee-8707-481a-965b-b35ee972f36a service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:51.773431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:51.774059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:51.774393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:51.774676 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:51.776930 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:51.791363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:51.794390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:51.819046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1380/2797] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:02:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:02:53.742918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:53.744412 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:53.757400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee034ecc-a30a-4b51-bba6-530b98333baa service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:53.762354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:53.762972 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:53.763247 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:53.763515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] PATH_INFO: `/v3/users/946a0fd33f754ce1aa534ab88dc95aa9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:53.764644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] RBAC: Authorizing `identity:delete_user(user_id=946a0fd33f754ce1aa534ab88dc95aa9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:53.766562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:53.812878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946a0fd33f754ce1aa534ab88dc95aa9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:53.820181 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946a0fd33f754ce1aa534ab88dc95aa9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:53.832281 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946a0fd33f754ce1aa534ab88dc95aa9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:53.839690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946a0fd33f754ce1aa534ab88dc95aa9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:53.840250 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946a0fd33f754ce1aa534ab88dc95aa9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:53.846258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1418/2798] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:02:53 2022] DELETE /identity/v3/users/946a0fd33f754ce1aa534ab88dc95aa9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:02:54.031588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:54.043326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e5f8f-fd2e-4283-a6ce-2ce475de271f service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:54.101954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:54.102585 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:54.102870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:54.103349 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] PATH_INFO: `/v3/projects/46ddf925b345471da94d05998a6a0540` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:54.104678 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ddf925b345471da94d05998a6a0540)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:54.106592 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:54.158869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ddf925b345471da94d05998a6a0540', 'request_id': 'req-66ebf30f-eb2d-41ce-9387-0011d3e859d7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.164936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ddf925b345471da94d05998a6a0540', 'request_id': 'req-66ebf30f-eb2d-41ce-9387-0011d3e859d7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.174449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ddf925b345471da94d05998a6a0540', 'request_id': 'req-66ebf30f-eb2d-41ce-9387-0011d3e859d7'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.186164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] The token cache is being invalidate because project 46ddf925b345471da94d05998a6a0540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:02:54.186459 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.187160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.189046 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1381/2799] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:02:54 2022] DELETE /identity/v3/projects/46ddf925b345471da94d05998a6a0540 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:02:54.192808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:54.194394 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:54.207499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2981714-7ace-407c-bcb0-049f107b012e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:54.219221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:54.273499 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:54.274128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:54.274403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:54.274673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] PATH_INFO: `/v3/users/a3081ea34e774ac89302eee4bff2cdc9` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:54.275788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3081ea34e774ac89302eee4bff2cdc9)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:54.277733 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:54.323670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3081ea34e774ac89302eee4bff2cdc9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.330310 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3081ea34e774ac89302eee4bff2cdc9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.342258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3081ea34e774ac89302eee4bff2cdc9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.350176 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3081ea34e774ac89302eee4bff2cdc9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.350815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-ae6b8c97-d4e5-4c76-93c0-313c354405e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3081ea34e774ac89302eee4bff2cdc9'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.356738 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1419/2800] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:02:54 2022] DELETE /identity/v3/users/a3081ea34e774ac89302eee4bff2cdc9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:02:54.521926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:02:54.523449 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:02:54.538784 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf30f-eb2d-41ce-9387-0011d3e859d7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:02:54.596340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:02:54.597363 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:02:54.597846 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:02:54.598828 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] PATH_INFO: `/v3/projects/1f97be6845804bd48b814faa1e32b34f` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:02:54.600866 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f97be6845804bd48b814faa1e32b34f)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:02:54.604288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:02:54.634580 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:02:54.659216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f97be6845804bd48b814faa1e32b34f', 'request_id': 'req-a13ac9b3-de5f-47c5-85ad-645d5caa7856'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.665201 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f97be6845804bd48b814faa1e32b34f', 'request_id': 'req-a13ac9b3-de5f-47c5-85ad-645d5caa7856'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.674005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f97be6845804bd48b814faa1e32b34f', 'request_id': 'req-a13ac9b3-de5f-47c5-85ad-645d5caa7856'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.685935 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] The token cache is being invalidate because project 1f97be6845804bd48b814faa1e32b34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:02:54.686301 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.686881 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:02:54.688657 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1382/2801] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:02:54 2022] DELETE /identity/v3/projects/1f97be6845804bd48b814faa1e32b34f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:03:04.353321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=46,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:03:04.682875 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:05:15.030451 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:05:15.030990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:05:15.030990 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:05:15.050049 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] MFA Rules not processed for user `1947f5ce1d794f05ae6272e8dba5d29c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:05:15.051258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:05:15.104430 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1420/2802] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:05:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:05:16.169571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:05:16.170977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:05:16.237433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ac9b3-de5f-47c5-85ad-645d5caa7856 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:05:16.242305 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:05:16.242925 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:05:16.243203 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:05:16.243470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:05:16.245183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:05:16.256253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:05:16.257860 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:05:16.268304 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1383/2803] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:05:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:05:25.097878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=25 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:05:26.263767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=32 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:35.911543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:35.913111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:35.928342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e58dd-2864-4af8-b210-ddc8881fd386 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:35.934089 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:35.934694 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:35.934973 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:35.935241 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:35.936962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:35.948448 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:35.950053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:35.960445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1421/2804] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:35.971670 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:35.973319 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:36.037968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae73d2-6614-4b2c-8443-02518b42ff30 service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.043437 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.044076 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.044361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.044668 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] PATH_INFO: `/v3/users/1947f5ce1d794f05ae6272e8dba5d29c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.045786 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1947f5ce1d794f05ae6272e8dba5d29c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.047757 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.094801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1947f5ce1d794f05ae6272e8dba5d29c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.101581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1947f5ce1d794f05ae6272e8dba5d29c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.113877 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1947f5ce1d794f05ae6272e8dba5d29c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.121041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1947f5ce1d794f05ae6272e8dba5d29c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.121662 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-3793c73c-b397-4fec-a31e-a7e0d1efeca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1947f5ce1d794f05ae6272e8dba5d29c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.127521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1384/2805] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:35 2022] DELETE /identity/v3/users/1947f5ce1d794f05ae6272e8dba5d29c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:06:36.304872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.318283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eca17-bac9-4969-b461-f5530647e99a service placement] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.376420 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.377080 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.377364 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.377640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] PATH_INFO: `/v3/projects/3218048072f048d9afc65ae138523bfd` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.378745 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3218048072f048d9afc65ae138523bfd)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.380724 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.426407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3218048072f048d9afc65ae138523bfd', 'request_id': 'req-66b772e4-0ac1-4629-bf7f-93747d309420'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.431820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3218048072f048d9afc65ae138523bfd', 'request_id': 'req-66b772e4-0ac1-4629-bf7f-93747d309420'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.438283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3218048072f048d9afc65ae138523bfd', 'request_id': 'req-66b772e4-0ac1-4629-bf7f-93747d309420'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.450140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] The token cache is being invalidate because project 3218048072f048d9afc65ae138523bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:36.450421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.451029 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-66b772e4-0ac1-4629-bf7f-93747d309420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:36.452473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1422/2806] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:36 2022] DELETE /identity/v3/projects/3218048072f048d9afc65ae138523bfd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:36.457324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.457615 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.457888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.476796 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:36.477977 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:36.534768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1385/2807] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:36.543321 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.543612 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.543888 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.569435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:36.570553 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:36.578860 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1423/2808] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:36.583136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.593542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf63e5-ca63-4dfd-9de8-9449838891b8 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.598061 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.598653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.598933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.599204 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.599873 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:06:36.600175 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.601803 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.613142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1386/2809] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:06:36.617269 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.626337 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6414b5-7384-4ce4-b1f9-0921a988e318 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.630889 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.631483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.631747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.632015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.632667 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.634518 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.663139 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1424/2810] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:36.668189 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.678422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1229cd-c96d-497e-b686-f5e38b65bfcb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.682042 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:36.734588 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.735185 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.735465 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.735738 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.736369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.738324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.760855 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1387/2811] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:36 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:36.765503 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.775814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfba0f-fdad-4718-863a-0758e6e559f5 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.780393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.781009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.781288 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.781558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.782185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.783763 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.793516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1425/2812] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:36.797186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.806954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98344e7-8f06-49fd-8dc8-3184cbefda78 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.812040 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.812685 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.812971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.813244 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.813904 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.815968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326425cf-77f5-4466-9a3e-73efef030eef admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.827308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1388/2813] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:06:36.831152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:36.841235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c5c38-d28c-4f6c-b19f-02846dc7f61c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:36.845899 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:36.846539 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.846813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.847082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] PATH_INFO: `/v3/projects/5ca77891b9a249d2aec642a497371b9b/users/d86b4e4a08e940889c286321af2796d2/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.861545 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca77891b9a249d2aec642a497371b9b, user_id=d86b4e4a08e940889c286321af2796d2, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:36.864164 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:36.877339 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1426/2814] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:36 2022] PUT /identity/v3/projects/5ca77891b9a249d2aec642a497371b9b/users/d86b4e4a08e940889c286321af2796d2/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:36.941938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:36.942226 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:36.942500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:36.970578 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] MFA Rules not processed for user `d86b4e4a08e940889c286321af2796d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:36.971788 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:37.034974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1389/2815] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:37.039548 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.053112 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:37.103230 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd8af8-d88e-44dc-be7d-d9ca48192078 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.108248 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.108868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.109144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.109408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.110030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.111900 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.139818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1427/2816] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:37 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:37.143771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.155227 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007449c8-4ee3-48a8-97d4-33fd2befb114 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.210295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.210895 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.211177 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.211452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.212085 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.214149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.235397 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1390/2817] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:37 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:37.239383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.250136 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208774b-dd63-4117-be28-4e460a096e45 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.254812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.255404 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.255686 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.255962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.256622 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.258227 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.270031 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1428/2818] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:37.273993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.280190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:37.284337 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c51105-2777-42bf-8031-b2244efaa782 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.290223 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.290864 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.291144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.291416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] PATH_INFO: `/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b/users/754c1c9e4d6941a885b8bdbe7ba59046/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.306625 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b535948445bd43d8a9a0b80d3c2a8f9b, user_id=754c1c9e4d6941a885b8bdbe7ba59046, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.309965 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.323047 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1391/2819] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:37 2022] PUT /identity/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b/users/754c1c9e4d6941a885b8bdbe7ba59046/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:37.326966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.336814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd3533-532a-4a94-b87c-3aa185e919da admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.392032 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.392552 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.392867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.393183 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.393818 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.395636 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ba9d6-c91f-422a-af53-8ae7d20bde5f admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.405005 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1429/2820] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:06:37.409022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.419596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c83c79-bc31-440f-928a-bb4e5bd23340 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.424149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.424816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.425102 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.425377 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] PATH_INFO: `/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b/users/754c1c9e4d6941a885b8bdbe7ba59046/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.427283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b535948445bd43d8a9a0b80d3c2a8f9b, user_id=754c1c9e4d6941a885b8bdbe7ba59046, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.429948 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.439932 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1392/2821] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:37 2022] PUT /identity/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b/users/754c1c9e4d6941a885b8bdbe7ba59046/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:37.449909 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b578b-5368-49bd-99c5-805401829519 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.450200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b578b-5368-49bd-99c5-805401829519 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.450480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b578b-5368-49bd-99c5-805401829519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.478011 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-143b578b-5368-49bd-99c5-805401829519 None None] MFA Rules not processed for user `754c1c9e4d6941a885b8bdbe7ba59046`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:37.479222 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-143b578b-5368-49bd-99c5-805401829519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:37.545015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1430/2822] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:37.556002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.558160 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:37.637056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747c5cf-cb5f-4afa-b5b0-9b6658c423af admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.640723 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.641318 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.641598 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.641869 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.650946 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.652311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba850a1-abce-42a7-84d4-5b98fb34a32d service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.661787 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1393/2823] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:37.689991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b578b-5368-49bd-99c5-805401829519 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.700088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:37.754270 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b578b-5368-49bd-99c5-805401829519 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.761562 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.762386 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.762670 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.762945 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] PATH_INFO: `/v3/users/d86b4e4a08e940889c286321af2796d2` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.764087 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86b4e4a08e940889c286321af2796d2)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:37.766038 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:37.810943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86b4e4a08e940889c286321af2796d2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:37.817653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86b4e4a08e940889c286321af2796d2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:37.829927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86b4e4a08e940889c286321af2796d2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:37.836369 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86b4e4a08e940889c286321af2796d2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:37.837030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86b4e4a08e940889c286321af2796d2'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:37.842780 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1431/2824] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:37 2022] DELETE /identity/v3/users/d86b4e4a08e940889c286321af2796d2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:37.847306 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.847614 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.848000 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:37.866750 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:37.867927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:37.925246 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1394/2825] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:06:37.935577 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:37.937036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:37.992150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228100-1d09-4bb1-bd51-dc02706bea8c admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:37.996345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:37.996971 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:37.997249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:37.997519 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:38.006333 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:38.007747 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:38.017053 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1432/2826] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:38.277661 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:38.279190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:38.293872 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038fda4-68a4-4266-8e55-4491c67a1d08 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:38.298639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:38.299379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:38.299731 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:38.300184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] PATH_INFO: `/v3/projects/5ca77891b9a249d2aec642a497371b9b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:38.301433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca77891b9a249d2aec642a497371b9b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:38.303667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:38.347483 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:38.350736 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca77891b9a249d2aec642a497371b9b', 'request_id': 'req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.358191 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca77891b9a249d2aec642a497371b9b', 'request_id': 'req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.365127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca77891b9a249d2aec642a497371b9b', 'request_id': 'req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.375801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] The token cache is being invalidate because project 5ca77891b9a249d2aec642a497371b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:38.376084 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.376717 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.378168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1395/2827] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:38 2022] DELETE /identity/v3/projects/5ca77891b9a249d2aec642a497371b9b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:38.382194 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:38.391862 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb7cad-ad10-4f4d-83ab-0292bd63bb86 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:38.421408 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:38.449190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:38.449808 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:38.450084 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:38.450354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] PATH_INFO: `/v3/users/754c1c9e4d6941a885b8bdbe7ba59046` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:38.451438 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754c1c9e4d6941a885b8bdbe7ba59046)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:38.453506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:38.497150 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754c1c9e4d6941a885b8bdbe7ba59046'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.504002 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754c1c9e4d6941a885b8bdbe7ba59046'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.515607 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754c1c9e4d6941a885b8bdbe7ba59046'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.523028 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754c1c9e4d6941a885b8bdbe7ba59046'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.523643 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-cc83df37-95b6-452c-89f1-13957fbb70c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754c1c9e4d6941a885b8bdbe7ba59046'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.529927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1433/2828] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:38 2022] DELETE /identity/v3/users/754c1c9e4d6941a885b8bdbe7ba59046 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:38.792071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:38.793769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:38.806663 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd8ae6-2de7-4d74-a5e8-9c21d1f6151d admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:38.862591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:38.863216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:38.863498 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:38.863770 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] PATH_INFO: `/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:38.864896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b535948445bd43d8a9a0b80d3c2a8f9b)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:38.866797 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:38.912980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b535948445bd43d8a9a0b80d3c2a8f9b', 'request_id': 'req-48a1b022-10a3-48c9-9219-72200a4a668d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.918603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b535948445bd43d8a9a0b80d3c2a8f9b', 'request_id': 'req-48a1b022-10a3-48c9-9219-72200a4a668d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.925476 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b535948445bd43d8a9a0b80d3c2a8f9b', 'request_id': 'req-48a1b022-10a3-48c9-9219-72200a4a668d'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.936271 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] The token cache is being invalidate because project b535948445bd43d8a9a0b80d3c2a8f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:38.936560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.937209 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-48a1b022-10a3-48c9-9219-72200a4a668d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:38.938666 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1396/2829] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:38 2022] DELETE /identity/v3/projects/b535948445bd43d8a9a0b80d3c2a8f9b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:38.943605 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:38.943896 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:38.944173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:38.964450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:38.965701 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:39.023672 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1434/2830] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:39.032778 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.033063 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.033334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.052554 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:39.053772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:39.063954 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1397/2831] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:39.068189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.078385 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928510e-a927-4ea8-9293-874f434ee325 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.082914 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.083500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.083776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.084044 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.084706 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:06:39.085006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.086619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.098189 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1435/2832] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:06:39.102279 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.111379 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d74d5-87b2-41b5-82cd-e3a32980196e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.116013 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.116618 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.116909 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.117179 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.117811 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.119692 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.147587 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1398/2833] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 05:06:39.151680 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.161532 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ee344-21b9-4689-a521-3e991594967a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.195265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:39.220954 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.221543 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.221815 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.222079 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.222704 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.224587 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.244357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1436/2834] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:39.248972 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.258982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fee9b-50ac-4ca5-a0de-8242364f2d31 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.264055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.264737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.265016 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.265283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.265923 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.267529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.277462 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1399/2835] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:39.281141 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.290905 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137690-386a-4353-b038-d30d22b52040 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.295455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.296036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.296311 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.296581 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.297292 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.298873 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011b1d-e2e4-4ea6-a489-866ab0e42216 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.309069 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1437/2836] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:39.312933 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.322628 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de671c2-d7ee-4d1a-88a1-2f0bf85433eb admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.328129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.328888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.329174 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.329452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] PATH_INFO: `/v3/projects/5d751c4437784ac1b6307a9549eee8ea/users/f086d42a07ee434bb1cee8aff7171d26/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.344308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d751c4437784ac1b6307a9549eee8ea, user_id=f086d42a07ee434bb1cee8aff7171d26, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.346989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.355982 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:39.361381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1400/2837] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:39 2022] PUT /identity/v3/projects/5d751c4437784ac1b6307a9549eee8ea/users/f086d42a07ee434bb1cee8aff7171d26/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:39.372239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.372529 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.372874 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.400567 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] MFA Rules not processed for user `f086d42a07ee434bb1cee8aff7171d26`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:39.401839 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:39.464619 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1438/2838] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:39.469296 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.528837 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b71b6-a3c2-41ec-a94b-d37011a377b2 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.533292 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.533898 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.534158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.534427 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.535054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.536975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.563783 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1401/2839] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 05:06:39.567938 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.578389 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49429502-9bf8-4e9a-b8ba-6fb881fb8db7 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.635314 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.636445 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.636755 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.637027 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.637666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.639569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.664342 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1439/2840] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:39.665595 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.675816 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c849-a958-4863-bac2-79806f10216f admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.680370 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.680997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.681274 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.681540 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.682169 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.683759 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.693968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1402/2841] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:39.698199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.708287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f85d22-7dd9-4cdb-b6e5-5109f300461e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.712791 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.713421 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.713695 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.714598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] PATH_INFO: `/v3/projects/4eda3dd58bb743c1b74338952a40a66c/users/8d6089f28c724af5ac6604a0493038f3/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.729490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eda3dd58bb743c1b74338952a40a66c, user_id=8d6089f28c724af5ac6604a0493038f3, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.732156 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.744890 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1440/2842] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:39 2022] PUT /identity/v3/projects/4eda3dd58bb743c1b74338952a40a66c/users/8d6089f28c724af5ac6604a0493038f3/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:39.750103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.762112 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf88ac-7749-44d0-a50a-d5216248113b admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.817393 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.817926 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.818220 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.818488 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.819120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.820741 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe1701-2e2c-4cc2-8bbf-d54e3f18c9a6 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.830795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1403/2843] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:39.835040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.845116 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6e8fa-d3af-485f-84fa-fa34cb79dbd9 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:39.849644 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:39.850278 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.850721 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.850842 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] PATH_INFO: `/v3/projects/4eda3dd58bb743c1b74338952a40a66c/users/8d6089f28c724af5ac6604a0493038f3/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.852718 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eda3dd58bb743c1b74338952a40a66c, user_id=8d6089f28c724af5ac6604a0493038f3, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:39.855572 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:39.861040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:39.866618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1441/2844] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:39 2022] PUT /identity/v3/projects/4eda3dd58bb743c1b74338952a40a66c/users/8d6089f28c724af5ac6604a0493038f3/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:39.878548 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:39.878850 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:39.879117 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:39.907038 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] MFA Rules not processed for user `8d6089f28c724af5ac6604a0493038f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:39.908251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:39.975452 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:39.986547 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1404/2845] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:39 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:39.997169 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:39.998852 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:40.055998 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bc78-05f7-4e98-bbd2-8e078963961e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:40.060020 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:40.060628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:40.060949 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:40.061221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:40.070137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:40.071492 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:40.081541 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1442/2846] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:39 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:40.157604 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:40.167384 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497e384-9599-45ad-a5d9-da7c80b6a357 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:40.171017 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:40.171617 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:40.171896 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:40.172168 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:40.229522 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:40.231217 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f359e5-a535-4a3c-a177-0cf51664ee8f service cinder] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:40.240055 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1405/2847] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:40 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:44.279334 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:44.339474 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42973ea-5cde-427b-b75f-bea0f36d50cf service cinder] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:44.343964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:44.344584 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:44.344894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:44.345168 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] PATH_INFO: `/v3/users/f086d42a07ee434bb1cee8aff7171d26` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:44.346280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f086d42a07ee434bb1cee8aff7171d26)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:44.348182 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:44.391609 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f086d42a07ee434bb1cee8aff7171d26'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.398776 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f086d42a07ee434bb1cee8aff7171d26'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.411996 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f086d42a07ee434bb1cee8aff7171d26'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.419425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f086d42a07ee434bb1cee8aff7171d26'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.420030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f086d42a07ee434bb1cee8aff7171d26'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.426103 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1443/2848] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:44 2022] DELETE /identity/v3/users/f086d42a07ee434bb1cee8aff7171d26 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:44.430539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:44.431031 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:44.431155 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:44.449496 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:44.450673 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:44.506315 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1406/2849] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:44.515800 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:44.517262 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:44.539560 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:44.576218 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c526e44-62f3-4380-a3d9-5a7a4b0871fb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:44.579878 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:44.580472 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:44.580788 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:44.581058 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:44.590345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:44.591756 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:44.601135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1444/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:44.862033 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:44.863509 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:44.878785 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83011d7-60a9-4a7d-9481-35dca007383e None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:44.884412 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:44.885051 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:44.885334 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:44.886129 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] PATH_INFO: `/v3/projects/5d751c4437784ac1b6307a9549eee8ea` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:44.887286 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d751c4437784ac1b6307a9549eee8ea)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:44.889237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:44.936568 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d751c4437784ac1b6307a9549eee8ea', 'request_id': 'req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.941971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d751c4437784ac1b6307a9549eee8ea', 'request_id': 'req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.949743 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d751c4437784ac1b6307a9549eee8ea', 'request_id': 'req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.960651 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] The token cache is being invalidate because project 5d751c4437784ac1b6307a9549eee8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:44.960959 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.961603 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:44.963070 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1407/2851] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:44 2022] DELETE /identity/v3/projects/5d751c4437784ac1b6307a9549eee8ea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:44.967652 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:44.980135 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd8c1-eed2-4307-b88e-7fc9c5f120ac service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.034498 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.035115 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.035395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.035663 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] PATH_INFO: `/v3/users/8d6089f28c724af5ac6604a0493038f3` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.036807 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6089f28c724af5ac6604a0493038f3)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.038709 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.087799 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6089f28c724af5ac6604a0493038f3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.095144 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6089f28c724af5ac6604a0493038f3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.107280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6089f28c724af5ac6604a0493038f3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.114439 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6089f28c724af5ac6604a0493038f3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.115166 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-c317d9c7-4c49-4d66-aefe-c1d1e27a5447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6089f28c724af5ac6604a0493038f3'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.120962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1445/2852] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:44 2022] DELETE /identity/v3/users/8d6089f28c724af5ac6604a0493038f3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:06:45.409809 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.411302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:45.424508 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd9d22-f0bf-4da8-98d2-cb6cd023e774 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.441422 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:45.486295 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.486853 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.487131 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.487400 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] PATH_INFO: `/v3/projects/4eda3dd58bb743c1b74338952a40a66c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.488494 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eda3dd58bb743c1b74338952a40a66c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.490410 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.535344 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eda3dd58bb743c1b74338952a40a66c', 'request_id': 'req-ee8db81f-bc6f-440a-83be-29244a2103a5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.540794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eda3dd58bb743c1b74338952a40a66c', 'request_id': 'req-ee8db81f-bc6f-440a-83be-29244a2103a5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.547916 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eda3dd58bb743c1b74338952a40a66c', 'request_id': 'req-ee8db81f-bc6f-440a-83be-29244a2103a5'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.558631 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] The token cache is being invalidate because project 4eda3dd58bb743c1b74338952a40a66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87798) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:45.558920 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.559570 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ee8db81f-bc6f-440a-83be-29244a2103a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:45.561056 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1408/2853] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:45 2022] DELETE /identity/v3/projects/4eda3dd58bb743c1b74338952a40a66c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:45.565872 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.566361 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.566490 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.587009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:45.588195 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:45.625097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:45.649762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1446/2854] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:45.659878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.660164 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.660429 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.682720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:45.683980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:45.691969 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1409/2855] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:45.696200 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.706500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610c124-b4cb-41eb-8f1a-b977796e0885 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.711199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.711797 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.712077 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.712347 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.713039 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87797) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:06:45.713341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.715621 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.727108 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1447/2856] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:06:45.731312 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.740463 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693607c1-4e55-41a9-9bf4-711c8308b677 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.745505 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.746087 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.746364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.746633 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.747253 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.749183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.775852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1410/2857] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:45 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:45.780199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.790157 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9faa41-b555-442f-94bf-c0e47ccc7ddd admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.851618 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.852223 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.852500 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.852789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.853425 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.855520 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.875942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1448/2858] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:45 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 05:06:45.880197 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.891041 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7a30b-3c48-4431-9962-d78b26c41967 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.895870 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.896484 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.896795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.897065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.897727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.899314 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.909727 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1411/2859] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:45.913390 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.923669 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f2fa9-5cb5-4006-a27e-d9b0c9fa7b4a admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.928199 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.928814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.929094 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.929367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.930050 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.931594 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ffea5-6937-4eb7-aeb0-27135e2c8d42 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.942315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1449/2860] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:45.946519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:45.956720 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d346a8-dc52-4667-ac4c-ba7c1fe3c725 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:45.961421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:45.962093 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:45.962373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:45.962650 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] PATH_INFO: `/v3/projects/01bf89b21e824ee29df060b2bd28dcca/users/f5308c1d931e4df6b3fa1385746b00f6/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:45.978372 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf89b21e824ee29df060b2bd28dcca, user_id=f5308c1d931e4df6b3fa1385746b00f6, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:45.981772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:45.996300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1412/2861] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:45 2022] PUT /identity/v3/projects/01bf89b21e824ee29df060b2bd28dcca/users/f5308c1d931e4df6b3fa1385746b00f6/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:46.008458 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.008784 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.009088 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.037173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] MFA Rules not processed for user `f5308c1d931e4df6b3fa1385746b00f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:46.038403 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:46.103985 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1450/2862] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:46.108575 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.157947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:46.176241 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4966fc-09eb-4106-89a5-96d030357bf0 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.181634 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.182238 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.182518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.182792 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.183424 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.185373 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.211921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1413/2863] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:46 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:46.216556 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.227974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b798bd-b96c-4fa8-8575-905778616876 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.261715 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:46.287013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.287628 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.287975 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.288243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] PATH_INFO: `/v3/users` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.288910 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.290820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.318983 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1451/2864] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:46 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:46.325092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.337549 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42481cf4-53c0-4ccb-abce-25b4fb7264d4 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.344518 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.345154 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.345438 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.345714 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.346366 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.347997 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.358804 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1414/2865] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:06:46.363249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.374811 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919968e-0c9d-453d-8ae4-c8c0896b9a0e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.379712 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.380393 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.380707 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.380984 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] PATH_INFO: `/v3/projects/7257d440c18a4a65b62d4f30ee107234/users/0673b4948ede49939e8e7f82f8d7201e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.396583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7257d440c18a4a65b62d4f30ee107234, user_id=0673b4948ede49939e8e7f82f8d7201e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.399280 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.412345 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1452/2866] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:46 2022] PUT /identity/v3/projects/7257d440c18a4a65b62d4f30ee107234/users/0673b4948ede49939e8e7f82f8d7201e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:06:46.416642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.426586 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041ca8-a6b7-43ac-b136-5a6828a927c3 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.482856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.483557 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.483726 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.483999 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.484658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.486288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee244498-f32f-45d0-903f-d690b3021d35 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.496340 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1415/2867] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:46.501700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.512974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4609e-6d3f-44c9-8bb5-89a7defa1520 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.517538 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.518177 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.518455 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.518722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] PATH_INFO: `/v3/projects/7257d440c18a4a65b62d4f30ee107234/users/0673b4948ede49939e8e7f82f8d7201e/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.520895 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7257d440c18a4a65b62d4f30ee107234, user_id=0673b4948ede49939e8e7f82f8d7201e, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.523601 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.534231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1453/2868] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:46 2022] PUT /identity/v3/projects/7257d440c18a4a65b62d4f30ee107234/users/0673b4948ede49939e8e7f82f8d7201e/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:46.544927 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.545216 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.545491 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.574681 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] MFA Rules not processed for user `0673b4948ede49939e8e7f82f8d7201e`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:46.575876 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:46.643308 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1416/2869] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:06:46.657825 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.660114 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:46.724746 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d035f-16c0-47d4-9ea3-e2d658e00b50 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.728812 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.729471 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.729751 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.730024 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.739034 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.740453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2931f4-b21f-4c10-85ef-8c367807e166 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:46.750243 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1454/2870] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:46 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:46.887814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:46.929900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:46.950644 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75576e65-a70f-49e2-8262-775c9e399c8b None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:46.955336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:46.955974 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:46.956257 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:46.956529 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] PATH_INFO: `/v3/users/f5308c1d931e4df6b3fa1385746b00f6` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:46.957684 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5308c1d931e4df6b3fa1385746b00f6)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:46.959601 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:47.006520 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5308c1d931e4df6b3fa1385746b00f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.013361 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5308c1d931e4df6b3fa1385746b00f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.025357 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5308c1d931e4df6b3fa1385746b00f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.032470 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5308c1d931e4df6b3fa1385746b00f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.033103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5308c1d931e4df6b3fa1385746b00f6'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.038942 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1417/2871] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:46 2022] DELETE /identity/v3/users/f5308c1d931e4df6b3fa1385746b00f6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:47.043892 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:47.044178 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:47.044484 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:47.063551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:47.064785 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:47.122966 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1455/2872] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:06:47.132704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:47.134250 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:47.191771 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0420b0-1867-470f-8e18-cca39ae6a5c8 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:47.195406 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:47.196006 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:47.196281 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:47.196562 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:47.205336 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:47.206760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:47.216364 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1418/2873] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:47.468516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:47.470030 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:47.482659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd2d04-5b82-489a-b633-a26ea826ec1e None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:47.487357 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:47.487981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:47.488374 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:47.488713 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] PATH_INFO: `/v3/projects/01bf89b21e824ee29df060b2bd28dcca` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:47.489820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bf89b21e824ee29df060b2bd28dcca)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:47.491758 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:47.523717 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:47.540868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bf89b21e824ee29df060b2bd28dcca', 'request_id': 'req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.547026 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bf89b21e824ee29df060b2bd28dcca', 'request_id': 'req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.553235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bf89b21e824ee29df060b2bd28dcca', 'request_id': 'req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.565009 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] The token cache is being invalidate because project 01bf89b21e824ee29df060b2bd28dcca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:47.565233 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.565851 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.567329 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1456/2874] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:47 2022] DELETE /identity/v3/projects/01bf89b21e824ee29df060b2bd28dcca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:06:47.571268 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:47.581158 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970477d-85e0-44a2-b12a-5638ad76c7c8 service neutron] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:47.635092 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:47.636517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:47.636826 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:47.637103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] PATH_INFO: `/v3/users/0673b4948ede49939e8e7f82f8d7201e` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:47.638208 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0673b4948ede49939e8e7f82f8d7201e)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:47.640170 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:47.642421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:47.686091 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0673b4948ede49939e8e7f82f8d7201e'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.692751 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0673b4948ede49939e8e7f82f8d7201e'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.704795 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0673b4948ede49939e8e7f82f8d7201e'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.711852 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0673b4948ede49939e8e7f82f8d7201e'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.712469 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-ae036b18-d1b9-43b5-a6cc-219d66954388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0673b4948ede49939e8e7f82f8d7201e'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:47.718975 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1419/2875] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:06:47 2022] DELETE /identity/v3/users/0673b4948ede49939e8e7f82f8d7201e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:47.984775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:47.986306 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:48.002354 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf69f-2f16-4bea-ab86-52ea6f9ed2f7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.064324 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.065004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.065282 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.065551 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] PATH_INFO: `/v3/projects/7257d440c18a4a65b62d4f30ee107234` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.066700 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7257d440c18a4a65b62d4f30ee107234)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.068726 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.113263 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7257d440c18a4a65b62d4f30ee107234', 'request_id': 'req-90fa4cf7-299f-47de-a669-c82bc87c46a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:48.118604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7257d440c18a4a65b62d4f30ee107234', 'request_id': 'req-90fa4cf7-299f-47de-a669-c82bc87c46a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:48.125231 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7257d440c18a4a65b62d4f30ee107234', 'request_id': 'req-90fa4cf7-299f-47de-a669-c82bc87c46a8'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:48.138485 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] The token cache is being invalidate because project 7257d440c18a4a65b62d4f30ee107234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:06:48.138770 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:48.139381 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-90fa4cf7-299f-47de-a669-c82bc87c46a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:06:48.140917 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1457/2876] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:06:47 2022] DELETE /identity/v3/projects/7257d440c18a4a65b62d4f30ee107234 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:48.146124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.146411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.146689 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.166054 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:48.167243 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:48.226378 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1420/2877] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:06:48.234224 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.234506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.234779 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.255239 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:48.256497 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:48.264855 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1458/2878] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:48.268938 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.278658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e296-dc03-446d-b07a-f6445ffecf8d None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.283219 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.283801 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.284078 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.284368 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.285026 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:06:48.285328 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.286940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.298703 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1421/2879] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:06:48.302929 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.312509 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52629d-744e-4eb6-97cb-f8f08bfcc966 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.317258 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.317847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.318126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.318395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.319070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.321025 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.347737 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1459/2880] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:06:48 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:48.351851 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.362222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad13c4-cff8-4481-a954-5dab822bb340 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.417710 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.418311 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.418591 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.418861 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.419482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.421458 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.438161 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:48.445519 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1422/2881] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:06:48 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:06:48.449673 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.459757 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebee96-22f9-4ccf-8423-a37fd0b88fbb admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.464363 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.464986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.465265 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.465534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.466173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.467778 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.477397 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1460/2882] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:06:48.481530 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.491127 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ea0cf-4a22-41e7-be3d-bccbb659c594 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.496302 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.496906 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.497183 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.497453 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.498077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.499653 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990f98a-82bd-4ca8-a81f-1f66b69175ad admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.510077 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1423/2883] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:06:48.513611 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.523596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec680c6-7a8d-49c3-a597-c18e07c50d15 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.528249 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.528919 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.529204 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.529569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] PATH_INFO: `/v3/projects/975cd6183a57461a9604d14868f48d6f/users/dadfd5efee64496d9d9322538fc4cde3/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.531887 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:06:48.547035 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=975cd6183a57461a9604d14868f48d6f, user_id=dadfd5efee64496d9d9322538fc4cde3, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.549720 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.562461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1461/2884] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:06:48 2022] PUT /identity/v3/projects/975cd6183a57461a9604d14868f48d6f/users/dadfd5efee64496d9d9322538fc4cde3/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:06:48.567371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13476-b5f1-4c1f-beef-698548954e91 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.567753 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13476-b5f1-4c1f-beef-698548954e91 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.568029 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae13476-b5f1-4c1f-beef-698548954e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.586355 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-fae13476-b5f1-4c1f-beef-698548954e91 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:48.587480 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fae13476-b5f1-4c1f-beef-698548954e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:48.643672 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1424/2885] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:48.653600 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:48.655285 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:48.710970 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddb4a8-affb-4c49-8722-e976043d1632 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:48.715015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:48.715602 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:48.715882 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:48.716149 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:48.724897 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:48.726305 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:48.736006 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1462/2886] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:49.349248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4d208-76b2-4c50-b83d-bf3140cab9ba None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:49.349495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4d208-76b2-4c50-b83d-bf3140cab9ba None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:49.349768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4d208-76b2-4c50-b83d-bf3140cab9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:49.377538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-52e4d208-76b2-4c50-b83d-bf3140cab9ba None None] MFA Rules not processed for user `dadfd5efee64496d9d9322538fc4cde3`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:49.378752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-52e4d208-76b2-4c50-b83d-bf3140cab9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:49.447616 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1425/2887] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:06:49.459697 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:06:49.461131 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:49.519775 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fc539-1946-4cce-9f97-a17ad0b80e4d service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:06:49.523400 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7cdae17359d4ba2ae149afbbd855ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:06:49.524004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:49.524284 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:49.524558 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:49.533076 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:06:49.534506 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc884d6-9ec8-48b4-894f-3bee4708e584 service cinder] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:06:49.543416 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1463/2888] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:06:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:06:50.898550 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76b510-01e4-4f91-866e-643790905533 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:06:50.899079 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76b510-01e4-4f91-866e-643790905533 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:06:50.899181 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76b510-01e4-4f91-866e-643790905533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:06:50.918425 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-9e76b510-01e4-4f91-866e-643790905533 None None] MFA Rules not processed for user `dadfd5efee64496d9d9322538fc4cde3`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:06:50.919542 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-9e76b510-01e4-4f91-866e-643790905533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:06:50.926215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1426/2889] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:06:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:06:59.539367 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:00.913564 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=68 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:05.137323 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:05.137820 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:05.137922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:05.156682 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] MFA Rules not processed for user `dadfd5efee64496d9d9322538fc4cde3`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:05.157877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:05.164868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1464/2890] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:15.151383 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:23.615799 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ccd7-928b-4b1b-b601-f797ab0e46ee None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:23.616343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ccd7-928b-4b1b-b601-f797ab0e46ee None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:23.616343 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ccd7-928b-4b1b-b601-f797ab0e46ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:23.665205 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-fa62ccd7-928b-4b1b-b601-f797ab0e46ee None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:23.666411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-fa62ccd7-928b-4b1b-b601-f797ab0e46ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:23.729752 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1427/2891] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:07:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:23.775435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:23.777407 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:23.868378 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb4ea3-9ab5-454a-a175-4d8049e2ef33 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:23.873395 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:23.873993 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:23.874273 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:23.874547 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:23.884494 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:23.886598 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:23.895226 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1465/2892] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:07:33.726847 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=40 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:33.891828 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=41 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:39.436819 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:39.437285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:39.437416 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:39.456526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:39.457739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:39.465358 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1428/2893] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 05:07:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:07:39.477047 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:39.487391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3124ad8-7d45-481e-ae1f-084fcd9eb395 service swift] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:39.492401 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:39.493018 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:39.493298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:39.493570 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:39.501312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:39.502867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:39.511384 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1466/2894] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:07:41.399679 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:41.413110 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e19d78-2e30-4911-831a-568692bf31f9 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:41.418233 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:41.418884 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.419162 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.419431 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] PATH_INFO: `/v3/users/dadfd5efee64496d9d9322538fc4cde3` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.420539 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dadfd5efee64496d9d9322538fc4cde3)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:41.422495 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:41.466936 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dadfd5efee64496d9d9322538fc4cde3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.473639 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dadfd5efee64496d9d9322538fc4cde3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.485921 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dadfd5efee64496d9d9322538fc4cde3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.492900 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dadfd5efee64496d9d9322538fc4cde3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.493512 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-1a4c9f0f-d5ed-41cb-9c9e-380da80badb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dadfd5efee64496d9d9322538fc4cde3'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.499166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1429/2895] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:07:41 2022] DELETE /identity/v3/users/dadfd5efee64496d9d9322538fc4cde3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:07:41.665583 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:41.678209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063c1d-400a-4925-8d9f-3effa4c4e4a7 service swift] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:41.734976 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:41.735596 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.735871 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.736140 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] PATH_INFO: `/v3/projects/975cd6183a57461a9604d14868f48d6f` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.737251 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=975cd6183a57461a9604d14868f48d6f)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:41.739128 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:41.783991 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '975cd6183a57461a9604d14868f48d6f', 'request_id': 'req-2da44ab4-cb51-42ec-9e9a-fa2485f97901'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.789341 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '975cd6183a57461a9604d14868f48d6f', 'request_id': 'req-2da44ab4-cb51-42ec-9e9a-fa2485f97901'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.797070 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '975cd6183a57461a9604d14868f48d6f', 'request_id': 'req-2da44ab4-cb51-42ec-9e9a-fa2485f97901'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.808988 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] The token cache is being invalidate because project 975cd6183a57461a9604d14868f48d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:07:41.809271 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.809879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-2da44ab4-cb51-42ec-9e9a-fa2485f97901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:07:41.811440 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1467/2896] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:07:41 2022] DELETE /identity/v3/projects/975cd6183a57461a9604d14868f48d6f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:07:41.817737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.818022 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.818298 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.839767 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:41.841002 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:41.897947 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1430/2897] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 20 05:07:41.906580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.906858 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.907151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.926681 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:41.927859 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:41.936097 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1468/2898] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:41.940159 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:41.949691 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45423c-7626-40e0-894d-d6cf5be319f2 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:41.954248 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:41.954839 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.955118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.955390 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] PATH_INFO: `/v3/domains` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.956023 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87798) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 20 05:07:41.956324 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:41.957968 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:41.969739 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1431/2899] 10.222.0.27 () {58 vars in 1153 bytes} [Tue Sep 20 05:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 20 05:07:41.974846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:41.985118 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bafa6-a6a3-4ecd-90bb-c0d14a07d864 None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:41.989829 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:41.990419 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:41.990693 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:41.990962 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:41.991589 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:41.993513 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.019190 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1469/2900] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:07:41 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:07:42.023768 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:42.035952 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6112c-4cc7-4856-bdd3-b17c751d7e36 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:42.081517 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:42.091813 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:42.092973 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.093288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.093669 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.094411 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:42.096404 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.117590 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1432/2901] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:07:42 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 20 05:07:42.121946 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:42.132557 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb1d9b-1ae5-4ab0-929e-9c749f51d98e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:42.137220 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:42.137813 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.138092 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.138365 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.138995 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:42.140573 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.152004 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1470/2902] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:07:42.156005 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:42.166072 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcaee4-e5d3-4969-9515-c60382e3eedf admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:42.171269 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:42.171858 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.172145 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.172418 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.173101 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:42.174687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed4806-e316-496f-9c96-5847756a0e67 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.185059 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1433/2903] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:07:42.189235 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:42.199060 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b32aa-279e-48c6-936b-a69d1a1629d6 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:42.203765 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:42.204441 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.204762 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.205040 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] PATH_INFO: `/v3/projects/15e8eefa98f44e8a871402c07adf11cf/users/cec291933cb14a0b8d6f8db32dbbe4b0/roles/3fbe9323075347109c2f57834701d0de` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.219837 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e8eefa98f44e8a871402c07adf11cf, user_id=cec291933cb14a0b8d6f8db32dbbe4b0, role_id=3fbe9323075347109c2f57834701d0de)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:42.222515 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.235702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1471/2904] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:07:42 2022] PUT /identity/v3/projects/15e8eefa98f44e8a871402c07adf11cf/users/cec291933cb14a0b8d6f8db32dbbe4b0/roles/3fbe9323075347109c2f57834701d0de => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:07:42.240831 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0f38-fb9e-4cf2-b5eb-fa64cd5497c4 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.241124 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0f38-fb9e-4cf2-b5eb-fa64cd5497c4 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.241401 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c0f38-fb9e-4cf2-b5eb-fa64cd5497c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.259915 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-732c0f38-fb9e-4cf2-b5eb-fa64cd5497c4 None None] MFA Rules not processed for user `095a92acb2e04f44b21db4de77e2a16a`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:42.261186 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-732c0f38-fb9e-4cf2-b5eb-fa64cd5497c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:42.319173 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1434/2905] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:42.329214 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:42.330870 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:42.345603 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:42.387665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555655da-78f4-431a-a06f-0ba02c7d7ee7 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:42.391302 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:42.391894 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:42.392173 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:42.392442 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:42.402163 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:42.403576 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:42.412702 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1472/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 05:07:46.111760 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.112283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.112283 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.147713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] MFA Rules not processed for user `cec291933cb14a0b8d6f8db32dbbe4b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:46.148940 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:46.233758 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1435/2907] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:46.237922 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.248659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a4019-373d-4ec0-9af6-de24a5101999 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.259902 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.261487 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.261981 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.262489 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.263396 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.265692 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.298252 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1473/2908] 10.222.0.27 () {60 vars in 1153 bytes} [Tue Sep 20 05:07:46 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:07:46.302152 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.314014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcbadf-0fdb-42d5-8b07-c5c90c658eb5 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.376354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.377007 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.377285 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.377553 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] PATH_INFO: `/v3/users` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.378199 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.380136 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.390989 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:46.408149 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1436/2909] 10.222.0.27 () {60 vars in 1144 bytes} [Tue Sep 20 05:07:46 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 20 05:07:46.411447 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.422129 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc3b4f-e211-4927-bf66-0b85bf999e36 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.430426 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.431604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.431604 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.431923 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.433366 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.435352 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.451074 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1474/2910] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 20 05:07:46.452288 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.463083 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6a5a8-fc15-4b47-a6f0-ce30f7cda25c admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.468089 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.468907 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.469323 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.469708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] PATH_INFO: `/v3/projects/fe559a311416450b8b8c2b7d14fc11ab/users/3b2d47b67435430ca028863b01d1cc8c/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.487525 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe559a311416450b8b8c2b7d14fc11ab, user_id=3b2d47b67435430ca028863b01d1cc8c, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.490254 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.508354 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1437/2911] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:07:46 2022] PUT /identity/v3/projects/fe559a311416450b8b8c2b7d14fc11ab/users/3b2d47b67435430ca028863b01d1cc8c/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:07:46.510391 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.522840 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d788b4-802f-45f9-bbba-b74084a5fc46 admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.591335 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.592974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.593819 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.594450 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.595327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.597036 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdd69bb-c7a0-451b-a7e2-5625dcc51e9b admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.612287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1475/2912] 10.222.0.27 () {58 vars in 1122 bytes} [Tue Sep 20 05:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 20 05:07:46.613386 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:46.631270 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c35aca-b7f0-4ef7-b69c-13db049025de admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:46.636646 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:46.637374 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:46.637599 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:46.637867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] PATH_INFO: `/v3/projects/fe559a311416450b8b8c2b7d14fc11ab/users/3b2d47b67435430ca028863b01d1cc8c/roles/6fa16de1683448f59199d3a6ed46c07c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:46.639776 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe559a311416450b8b8c2b7d14fc11ab, user_id=3b2d47b67435430ca028863b01d1cc8c, role_id=6fa16de1683448f59199d3a6ed46c07c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:46.642500 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:46.656967 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1438/2913] 10.222.0.27 () {60 vars in 1483 bytes} [Tue Sep 20 05:07:46 2022] PUT /identity/v3/projects/fe559a311416450b8b8c2b7d14fc11ab/users/3b2d47b67435430ca028863b01d1cc8c/roles/6fa16de1683448f59199d3a6ed46c07c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:07:51.240772 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8fd8-e3cb-42db-88f1-28b74bafa896 None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:51.241283 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8fd8-e3cb-42db-88f1-28b74bafa896 None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:51.241387 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8fd8-e3cb-42db-88f1-28b74bafa896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:51.279041 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-eb9a8fd8-e3cb-42db-88f1-28b74bafa896 None None] MFA Rules not processed for user `3b2d47b67435430ca028863b01d1cc8c`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:51.281676 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-eb9a8fd8-e3cb-42db-88f1-28b74bafa896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:51.383986 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1476/2914] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:07:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:51.402888 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:51.405307 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:51.505441 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473f353-5ea1-469b-9f6f-dad40c4d29bd admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:51.509114 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:51.509719 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:51.509993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:51.510261 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:51.575962 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:51.577461 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:51.586769 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1439/2915] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:07:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 05:07:51.730769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2e910-d63b-4fb9-9933-84b16fd8375e None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:51.731568 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2e910-d63b-4fb9-9933-84b16fd8375e None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:51.733486 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2e910-d63b-4fb9-9933-84b16fd8375e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:51.743969 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:51.758653 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-47c2e910-d63b-4fb9-9933-84b16fd8375e None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:07:51.759958 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-47c2e910-d63b-4fb9-9933-84b16fd8375e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:51.820075 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1477/2916] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:07:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:07:51.833194 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:07:51.834658 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:07:51.891713 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:07:51.913333 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b16359-550e-453d-86ac-501dacc8531a service glance] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:07:51.919053 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:07:51.919683 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:07:51.919964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:07:51.920237 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:07:51.928867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:07:51.930466 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:07:51.939165 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1440/2917] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:07:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:08:00.507202 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:00.507787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:00.507787 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:00.528217 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] MFA Rules not processed for user `cec291933cb14a0b8d6f8db32dbbe4b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:08:00.529461 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:00.536174 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1478/2918] 10.222.0.27 () {58 vars in 958 bytes} [Tue Sep 20 05:08:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:08:00.571833 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:00.581686 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f6608-9b30-40bf-a784-0f4408c93432 service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:00.585264 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef0eb7d5e5b1416b8018e1a5c49a33e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:00.585844 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:00.586121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:00.586391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:00.588065 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:00.671706 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:00.674048 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:00.688120 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1441/2919] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:00.908480 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:00.909875 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:00.966615 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf84ae-9467-4526-a7cf-7357a2e4f8ae None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:00.970126 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:00.970722 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:00.970999 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:00.971268 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:00.973015 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:00.983023 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:00.984435 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19f95c-cdf0-40eb-87e3-c36e494bbd13 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:00.992948 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1479/2920] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:01.298482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:01.299862 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:01.376166 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608686-69f1-4fa3-a162-c2b4ec91eeda service glance] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:01.381167 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:01.381772 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:01.382060 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:01.382335 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:01.384030 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:01.396118 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:01.397734 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:01.408433 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1442/2921] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:02.689942 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] REQUEST_METHOD: `POST` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:02.690477 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:02.690580 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:02.710659 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.auth.core [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87797) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:08:02.711868 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:02.719645 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1480/2922] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:08:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:08:02.732697 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:02.744081 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e88551-0f98-4d12-a5bb-0f1c3dee27cb service placement] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:02.749190 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:02.749893 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:02.750258 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:02.750602 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:02.758737 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:02.760321 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:02.769039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1443/2923] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:08:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 20 05:08:10.627608 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:10.637690 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8d6d0-ccce-42e9-b30e-32f7c6f32a3b None None] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:10.641287 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:10.641877 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:10.642152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:10.642424 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:10.644120 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:10.654081 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:10.655616 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:10.664304 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1481/2924] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:11.293479 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:11.295326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:11.311632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94cdb4-29eb-4b3e-9201-5c01fd82e730 service swift] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:11.317103 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:11.317704 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:11.317981 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:11.318251 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:11.319964 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:11.342521 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:11.344130 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:11.355667 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1444/2925] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:17.423847 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:17.439664 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcda7-442e-4eff-9515-46d7f4b6c072 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:17.445221 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:17.445814 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:17.446091 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:17.446358 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:17.448666 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:17.460891 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:17.462452 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:17.472964 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1482/2926] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:21.350184 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=88 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:08:27.468516 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=67 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:08:27.998153 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:28.007432 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9ac40-bcd6-44c7-9a5e-27fb93919779 service nova] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:28.010993 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:28.011581 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:28.011856 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:28.012122 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:28.020746 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:28.022421 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a24ad0-5914-4684-a8b8-b8285041fd83 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:28.031963 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1445/2927] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:29.796209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:08:29.809013 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807008f1-8cff-4172-8ebb-50ee28129d49 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:08:29.816215 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9809a4423cfb4ddda8d03e663f81b060', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:08:29.816953 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:08:29.817275 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:08:29.817638 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:08:29.819428 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:08:29.832147 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:08:29.833974 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:08:29.846137 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1483/2928] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:08:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:08:38.027978 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=6 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:08:39.841151 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=6 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:09:47.288300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:47.288794 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:47.288908 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:47.308700 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] MFA Rules not processed for user `aab2ee7790444760b8333226be499621`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:09:47.309903 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:47.317863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1446/2929] 10.222.0.27 () {58 vars in 967 bytes} [Tue Sep 20 05:09:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:09:47.357665 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:47.369833 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969725-d0cf-4e04-b18c-e95481316db8 service nova] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:47.374928 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7355daeb39aa4d8c8c1eb698fede46f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:47.375517 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:47.375796 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:47.376065 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:47.383886 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:47.385483 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:47.393963 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1484/2930] 10.222.0.27 () {60 vars in 1384 bytes} [Tue Sep 20 05:09:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 20 05:09:50.958176 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:50.970014 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7277200-42bc-4893-b8df-ccb3145574bc None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:50.973687 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:50.974391 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:50.974939 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:50.975300 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:50.977754 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:50.995371 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:50.997818 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf52860-3c8e-4366-a4c1-bb72e4753455 service neutron] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:51.017381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1447/2931] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:09:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 20 05:09:51.027766 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:51.041834 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ecd68-6f4f-43d8-b8f0-917988fac3d6 service swift] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:51.045879 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:51.046789 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:51.047320 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:51.047590 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:51.049867 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] Could not find project: 2e65f62fae68435ea0304bebc2a21b90.: keystone.exception.ProjectNotFound: Could not find project: 2e65f62fae68435ea0304bebc2a21b90. Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 20 05:09:51.064312 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 05:09:51.067769 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 20 05:09:51.069014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.get_or_create( Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application with Lock( Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._enter() Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.creator() Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application created_value = creator( Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 20 05:09:51.070244 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 20 05:09:51.071417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2e65f62fae68435ea0304bebc2a21b90. Sep 20 05:09:51.071417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: ERROR keystone.server.flask.application Sep 20 05:09:51.071417 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1485/2932] 10.222.0.27 () {60 vars in 1396 bytes} [Tue Sep 20 05:09:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 20 05:09:51.103766 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] REQUEST_METHOD: `POST` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:51.104151 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:51.104510 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:51.149144 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.auth.core [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] MFA Rules not processed for user `9809a4423cfb4ddda8d03e663f81b060`. Rule list: `[]` (Enabled: `True`). {{(pid=87798) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 20 05:09:51.150814 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:51.160773 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1448/2933] 10.222.0.27 () {58 vars in 1009 bytes} [Tue Sep 20 05:09:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 20 05:09:51.177208 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:51.186327 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfaf02-dd38-44c8-9a46-ecd0f29561e6 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:51.191315 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fe2203ea1f94623956a215190e4d79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:51.191918 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] REQUEST_METHOD: `GET` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:51.192198 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:51.192473 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:51.203161 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:51.204613 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:51.216911 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1486/2934] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:09:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:09:54.048381 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:54.120277 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ed2b-61b9-45ef-8ec1-f2f3263eba68 None None] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:54.128482 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:54.129655 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:54.129971 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:54.130256 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] PATH_INFO: `/v3/users/cec291933cb14a0b8d6f8db32dbbe4b0` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:54.131552 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec291933cb14a0b8d6f8db32dbbe4b0)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:54.133596 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:54.184293 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec291933cb14a0b8d6f8db32dbbe4b0'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.191632 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec291933cb14a0b8d6f8db32dbbe4b0'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.203338 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec291933cb14a0b8d6f8db32dbbe4b0'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.210867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec291933cb14a0b8d6f8db32dbbe4b0'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.211526 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec291933cb14a0b8d6f8db32dbbe4b0'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.217230 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1449/2935] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:09:54 2022] DELETE /identity/v3/users/cec291933cb14a0b8d6f8db32dbbe4b0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:09:54.394565 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:54.406162 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726162d-21bd-464b-8da5-43d623ce4f73 service neutron] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:54.464298 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:54.464927 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:54.465209 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:54.465482 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] PATH_INFO: `/v3/projects/15e8eefa98f44e8a871402c07adf11cf` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:54.466582 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e8eefa98f44e8a871402c07adf11cf)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:54.468514 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:54.513414 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e8eefa98f44e8a871402c07adf11cf', 'request_id': 'req-b1458b41-3424-47bc-bedf-80be983fee4e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.518801 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e8eefa98f44e8a871402c07adf11cf', 'request_id': 'req-b1458b41-3424-47bc-bedf-80be983fee4e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.524978 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e8eefa98f44e8a871402c07adf11cf', 'request_id': 'req-b1458b41-3424-47bc-bedf-80be983fee4e'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.537111 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] The token cache is being invalidate because project 15e8eefa98f44e8a871402c07adf11cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:09:54.537410 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.538082 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.539534 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1487/2936] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:09:54 2022] DELETE /identity/v3/projects/15e8eefa98f44e8a871402c07adf11cf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:09:54.543492 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:54.544980 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:54.558215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb1635-f31c-4633-9767-9c0be7a127e7 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:54.595039 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:09:54.615215 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:54.615840 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:54.616121 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:54.616398 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] PATH_INFO: `/v3/users/3b2d47b67435430ca028863b01d1cc8c` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:54.617571 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2d47b67435430ca028863b01d1cc8c)` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:54.619503 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:54.665835 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2d47b67435430ca028863b01d1cc8c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.674443 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2d47b67435430ca028863b01d1cc8c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.687326 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2d47b67435430ca028863b01d1cc8c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.696282 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2d47b67435430ca028863b01d1cc8c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.696945 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.notifications [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2d47b67435430ca028863b01d1cc8c'} {{(pid=87798) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:54.703369 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1450/2937] 10.222.0.27 () {58 vars in 1224 bytes} [Tue Sep 20 05:09:54 2022] DELETE /identity/v3/users/3b2d47b67435430ca028863b01d1cc8c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 20 05:09:54.876191 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Authenticating user token {{(pid=87797) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:09:54.877767 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.fernet_utils [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87797) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:09:54.892373 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1458b41-3424-47bc-bedf-80be983fee4e admin admin] Validating token access rules against request {{(pid=87797) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:09:54.942940 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:09:54.950846 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '095a92acb2e04f44b21db4de77e2a16a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '612c023f1ed44b9f936be9c752c292ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87797) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:09:54.951640 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:09:54.952185 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:09:54.953152 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] PATH_INFO: `/v3/projects/fe559a311416450b8b8c2b7d14fc11ab` {{(pid=87797) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:09:54.954453 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe559a311416450b8b8c2b7d14fc11ab)` {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:09:54.956349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] RBAC: Authorization granted {{(pid=87797) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:09:55.001095 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe559a311416450b8b8c2b7d14fc11ab', 'request_id': 'req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:55.006349 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe559a311416450b8b8c2b7d14fc11ab', 'request_id': 'req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:55.012893 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe559a311416450b8b8c2b7d14fc11ab', 'request_id': 'req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4'} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:55.025661 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] The token cache is being invalidate because project fe559a311416450b8b8c2b7d14fc11ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87797) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 20 05:09:55.025943 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:55.026566 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG keystone.notifications [None req-f2ca62a9-1d94-4ada-8253-944ffb43e9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87797) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 20 05:09:55.028014 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: [pid: 87797|app: 0|req: 1488/2938] 10.222.0.27 () {58 vars in 1233 bytes} [Tue Sep 20 05:09:54 2022] DELETE /identity/v3/projects/fe559a311416450b8b8c2b7d14fc11ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 20 05:10:04.699708 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:10:05.022569 dv-cinder-756710-7-iscsi devstack@keystone.service[87797]: DEBUG dbcounter [-] [87797] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87797) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 20 05:11:17.215642 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Authenticating user token {{(pid=87798) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 20 05:11:17.217071 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:11:17.352538 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00732e34-2a74-428c-bc74-1e00e8edb8d1 admin admin] Validating token access rules against request {{(pid=87798) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 20 05:11:17.412024 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '097f3a7d28294befa98d30e220e01e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a04e187c73948d6947123e45487906b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87798) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 20 05:11:17.412560 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] REQUEST_METHOD: `GET` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 20 05:11:17.412867 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] SCRIPT_NAME: `/identity` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 20 05:11:17.413142 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87798) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 20 05:11:17.414863 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.fernet_utils [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87798) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 20 05:11:17.425878 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 20 05:11:17.427487 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8be3c9-e435-4199-b14c-a09be42c8ebb service placement] RBAC: Authorization granted {{(pid=87798) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 20 05:11:17.437417 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: [pid: 87798|app: 0|req: 1451/2939] 10.222.0.27 () {60 vars in 1365 bytes} [Tue Sep 20 05:11:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 20 05:11:27.433222 dv-cinder-756710-7-iscsi devstack@keystone.service[87798]: DEBUG dbcounter [-] [87798] Writing DB stats keystone:SELECT=89 {{(pid=87798) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}